2015-10-14 18:42:55,545 INFO [main] hbase.IntegrationTestsDriver: Found 1 integration tests to run: 2015-10-14 18:42:55,546 INFO [main] hbase.IntegrationTestsDriver: class org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient .2015-10-14 18:42:55,648 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting test 2015-10-14 18:42:55,821 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50146 2015-10-14 18:42:55,895 INFO [Time-limited test] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties 2015-10-14 18:42:56,306 INFO [Time-limited test] impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2015-10-14 18:42:56,306 INFO [Time-limited test] impl.MetricsSystemImpl: HBase metrics system started 2015-10-14 18:42:56,430 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,430 INFO [RpcServer.listener,port=50146] ipc.RpcServer: RpcServer.listener,port=50146: starting 2015-10-14 18:42:56,431 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50146 2015-10-14 18:42:56,433 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50147 2015-10-14 18:42:56,433 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,433 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50147 2015-10-14 18:42:56,433 INFO [RpcServer.listener,port=50147] ipc.RpcServer: RpcServer.listener,port=50147: starting 2015-10-14 18:42:56,435 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50148 2015-10-14 18:42:56,435 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,435 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50148 2015-10-14 18:42:56,435 INFO [RpcServer.listener,port=50148] ipc.RpcServer: RpcServer.listener,port=50148: starting 2015-10-14 18:42:56,445 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50149 2015-10-14 18:42:56,446 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,446 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50149 2015-10-14 18:42:56,446 INFO [RpcServer.listener,port=50149] ipc.RpcServer: RpcServer.listener,port=50149: starting 2015-10-14 18:42:56,447 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50150 2015-10-14 18:42:56,448 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,448 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50150 2015-10-14 18:42:56,448 INFO [RpcServer.listener,port=50150] ipc.RpcServer: RpcServer.listener,port=50150: starting 2015-10-14 18:42:56,449 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50151 2015-10-14 18:42:56,450 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,450 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50151 2015-10-14 18:42:56,450 INFO [RpcServer.listener,port=50151] ipc.RpcServer: RpcServer.listener,port=50151: starting 2015-10-14 18:42:56,452 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50152 2015-10-14 18:42:56,452 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,452 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50152 2015-10-14 18:42:56,452 INFO [RpcServer.listener,port=50152] ipc.RpcServer: RpcServer.listener,port=50152: starting 2015-10-14 18:42:56,454 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50153 2015-10-14 18:42:56,454 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,454 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50153 2015-10-14 18:42:56,454 INFO [RpcServer.listener,port=50153] ipc.RpcServer: RpcServer.listener,port=50153: starting 2015-10-14 18:42:56,456 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50154 2015-10-14 18:42:56,457 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,457 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50154 2015-10-14 18:42:56,457 INFO [RpcServer.listener,port=50154] ipc.RpcServer: RpcServer.listener,port=50154: starting 2015-10-14 18:42:56,458 INFO [Time-limited test] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50155 2015-10-14 18:42:56,459 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,459 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50155 2015-10-14 18:42:56,459 INFO [RpcServer.listener,port=50155] ipc.RpcServer: RpcServer.listener,port=50155: starting 2015-10-14 18:42:56,522 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_0_ 2015-10-14 18:42:56,522 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_1_ 2015-10-14 18:42:56,523 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_2_ 2015-10-14 18:42:56,523 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_3_ 2015-10-14 18:42:56,523 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_4_ 2015-10-14 18:42:56,523 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_5_ 2015-10-14 18:42:56,523 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_6_ 2015-10-14 18:42:56,523 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_7_ 2015-10-14 18:42:56,523 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_8_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_9_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_10_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_11_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_12_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_13_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_14_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_15_ 2015-10-14 18:42:56,524 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_16_ 2015-10-14 18:42:56,525 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_17_ 2015-10-14 18:42:56,525 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_18_ 2015-10-14 18:42:56,525 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_19_ 2015-10-14 18:42:56,525 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_20_ 2015-10-14 18:42:56,525 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_21_ 2015-10-14 18:42:56,525 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_22_ 2015-10-14 18:42:56,525 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_23_ 2015-10-14 18:42:56,526 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_24_ 2015-10-14 18:42:56,526 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_25_ 2015-10-14 18:42:56,526 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_26_ 2015-10-14 18:42:56,526 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_27_ 2015-10-14 18:42:56,526 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_28_ 2015-10-14 18:42:56,526 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting client: client_29_ 2015-10-14 18:42:56,526 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Starting MiniChaosMonkey 2015-10-14 18:42:56,734 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50156 2015-10-14 18:42:56,734 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,734 INFO [RpcServer.listener,port=50156] ipc.RpcServer: RpcServer.listener,port=50156: starting 2015-10-14 18:42:56,734 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50156 2015-10-14 18:42:56,777 WARN [Thread-13] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2015-10-14 18:42:56,839 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50153 2015-10-14 18:42:56,840 INFO [Thread-43] ipc.RpcServer: Stopping server on 50153 2015-10-14 18:42:56,840 INFO [RpcServer.listener,port=50153] ipc.RpcServer: RpcServer.listener,port=50153: stopping 2015-10-14 18:42:56,841 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:42:56,841 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50153 2015-10-14 18:42:56,841 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:42:56,909 WARN [Thread-37] ipc.IntegrationTestRpcClient: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused: /127.0.0.1:50153 2015-10-14 18:42:56,909 WARN [Thread-35] ipc.IntegrationTestRpcClient: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused: /127.0.0.1:50153 2015-10-14 18:42:56,948 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50168 2015-10-14 18:42:56,949 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:56,949 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50168 2015-10-14 18:42:56,949 INFO [RpcServer.listener,port=50168] ipc.RpcServer: RpcServer.listener,port=50168: starting 2015-10-14 18:42:57,087 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50169 2015-10-14 18:42:57,098 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:57,098 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50169 2015-10-14 18:42:57,098 INFO [RpcServer.listener,port=50169] ipc.RpcServer: RpcServer.listener,port=50169: starting 2015-10-14 18:42:57,236 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50151 2015-10-14 18:42:57,236 INFO [Thread-43] ipc.RpcServer: Stopping server on 50151 2015-10-14 18:42:57,236 INFO [RpcServer.listener,port=50151] ipc.RpcServer: RpcServer.listener,port=50151: stopping 2015-10-14 18:42:57,242 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:42:57,242 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:42:57,242 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50151 2015-10-14 18:42:57,244 WARN [AsyncRpcChannel-pool2-t3] channel.DefaultChannelPipeline: An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception. java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) at io.netty.buffer.UnpooledUnsafeDirectByteBuf.setBytes(UnpooledUnsafeDirectByteBuf.java:447) at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:881) at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:242) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:119) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) at java.lang.Thread.run(Thread.java:745) 2015-10-14 18:42:57,383 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50172 2015-10-14 18:42:57,386 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:57,386 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50172 2015-10-14 18:42:57,386 INFO [RpcServer.listener,port=50172] ipc.RpcServer: RpcServer.listener,port=50172: starting 2015-10-14 18:42:57,509 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50174 2015-10-14 18:42:57,523 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:57,523 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50174 2015-10-14 18:42:57,523 INFO [RpcServer.listener,port=50174] ipc.RpcServer: RpcServer.listener,port=50174: starting 2015-10-14 18:42:57,640 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50154 2015-10-14 18:42:57,640 INFO [Thread-43] ipc.RpcServer: Stopping server on 50154 2015-10-14 18:42:57,640 INFO [RpcServer.listener,port=50154] ipc.RpcServer: RpcServer.listener,port=50154: stopping 2015-10-14 18:42:57,659 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:42:57,659 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:42:57,659 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50154 2015-10-14 18:42:57,773 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50176 2015-10-14 18:42:57,775 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:57,775 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50176 2015-10-14 18:42:57,775 INFO [RpcServer.listener,port=50176] ipc.RpcServer: RpcServer.listener,port=50176: starting 2015-10-14 18:42:57,890 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50177 2015-10-14 18:42:57,891 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:57,891 INFO [RpcServer.listener,port=50177] ipc.RpcServer: RpcServer.listener,port=50177: starting 2015-10-14 18:42:57,891 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50177 2015-10-14 18:42:58,115 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50180 2015-10-14 18:42:58,117 INFO [RpcServer.listener,port=50180] ipc.RpcServer: RpcServer.listener,port=50180: starting 2015-10-14 18:42:58,117 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50180 2015-10-14 18:42:58,117 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:58,230 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50182 2015-10-14 18:42:58,232 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:58,232 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50182 2015-10-14 18:42:58,232 INFO [RpcServer.listener,port=50182] ipc.RpcServer: RpcServer.listener,port=50182: starting 2015-10-14 18:42:58,468 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50184 2015-10-14 18:42:58,470 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:58,470 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50184 2015-10-14 18:42:58,470 INFO [RpcServer.listener,port=50184] ipc.RpcServer: RpcServer.listener,port=50184: starting 2015-10-14 18:42:58,574 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50184 2015-10-14 18:42:58,574 INFO [Thread-43] ipc.RpcServer: Stopping server on 50184 2015-10-14 18:42:58,574 INFO [RpcServer.listener,port=50184] ipc.RpcServer: RpcServer.listener,port=50184: stopping 2015-10-14 18:42:58,581 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:42:58,581 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:42:58,581 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50184 2015-10-14 18:42:58,897 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50185 2015-10-14 18:42:58,899 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:58,899 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50185 2015-10-14 18:42:58,899 INFO [RpcServer.listener,port=50185] ipc.RpcServer: RpcServer.listener,port=50185: starting 2015-10-14 18:42:59,036 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50187 2015-10-14 18:42:59,038 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:59,038 INFO [RpcServer.listener,port=50187] ipc.RpcServer: RpcServer.listener,port=50187: starting 2015-10-14 18:42:59,038 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50187 2015-10-14 18:42:59,138 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50168 2015-10-14 18:42:59,139 INFO [Thread-43] ipc.RpcServer: Stopping server on 50168 2015-10-14 18:42:59,141 INFO [RpcServer.listener,port=50168] ipc.RpcServer: RpcServer.listener,port=50168: stopping 2015-10-14 18:42:59,147 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:42:59,147 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50168 2015-10-14 18:42:59,147 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:42:59,264 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50189 2015-10-14 18:42:59,266 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:59,266 INFO [RpcServer.listener,port=50189] ipc.RpcServer: RpcServer.listener,port=50189: starting 2015-10-14 18:42:59,266 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50189 2015-10-14 18:42:59,381 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50191 2015-10-14 18:42:59,383 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:59,383 INFO [RpcServer.listener,port=50191] ipc.RpcServer: RpcServer.listener,port=50191: starting 2015-10-14 18:42:59,383 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50191 2015-10-14 18:42:59,629 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50193 2015-10-14 18:42:59,647 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:59,647 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50193 2015-10-14 18:42:59,647 INFO [RpcServer.listener,port=50193] ipc.RpcServer: RpcServer.listener,port=50193: starting 2015-10-14 18:42:59,749 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50185 2015-10-14 18:42:59,749 INFO [Thread-43] ipc.RpcServer: Stopping server on 50185 2015-10-14 18:42:59,749 INFO [RpcServer.listener,port=50185] ipc.RpcServer: RpcServer.listener,port=50185: stopping 2015-10-14 18:42:59,780 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:42:59,780 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50185 2015-10-14 18:42:59,780 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:42:59,892 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50195 2015-10-14 18:42:59,893 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:42:59,893 INFO [RpcServer.listener,port=50195] ipc.RpcServer: RpcServer.listener,port=50195: starting 2015-10-14 18:42:59,893 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50195 2015-10-14 18:42:59,998 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50152 2015-10-14 18:42:59,998 INFO [Thread-43] ipc.RpcServer: Stopping server on 50152 2015-10-14 18:42:59,999 INFO [RpcServer.listener,port=50152] ipc.RpcServer: RpcServer.listener,port=50152: stopping 2015-10-14 18:43:00,011 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:00,011 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:00,011 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50152 2015-10-14 18:43:00,115 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50149 2015-10-14 18:43:00,116 INFO [Thread-43] ipc.RpcServer: Stopping server on 50149 2015-10-14 18:43:00,116 INFO [RpcServer.listener,port=50149] ipc.RpcServer: RpcServer.listener,port=50149: stopping 2015-10-14 18:43:00,126 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50149 2015-10-14 18:43:00,126 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:00,126 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:00,230 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50197 2015-10-14 18:43:00,231 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:00,232 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50197 2015-10-14 18:43:00,232 INFO [RpcServer.listener,port=50197] ipc.RpcServer: RpcServer.listener,port=50197: starting 2015-10-14 18:43:00,338 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50198 2015-10-14 18:43:00,339 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50198 2015-10-14 18:43:00,339 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:00,339 INFO [RpcServer.listener,port=50198] ipc.RpcServer: RpcServer.listener,port=50198: starting 2015-10-14 18:43:00,442 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50169 2015-10-14 18:43:00,443 INFO [Thread-43] ipc.RpcServer: Stopping server on 50169 2015-10-14 18:43:00,443 INFO [RpcServer.listener,port=50169] ipc.RpcServer: RpcServer.listener,port=50169: stopping 2015-10-14 18:43:00,460 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:00,460 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50169 2015-10-14 18:43:00,460 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:00,666 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50189 2015-10-14 18:43:00,666 INFO [Thread-43] ipc.RpcServer: Stopping server on 50189 2015-10-14 18:43:00,666 INFO [RpcServer.listener,port=50189] ipc.RpcServer: RpcServer.listener,port=50189: stopping 2015-10-14 18:43:00,677 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:00,677 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50189 2015-10-14 18:43:00,677 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:00,714 WARN [FifoRpcScheduler.handler23-thread-1] ipc.RpcServer: FifoRpcScheduler.handler23-thread-1: caught a ClosedChannelException, this means that the server /127.0.0.1:50189 was processing a request but the client went away. The error message was: null 2015-10-14 18:43:00,779 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50150 2015-10-14 18:43:00,780 INFO [Thread-43] ipc.RpcServer: Stopping server on 50150 2015-10-14 18:43:00,780 INFO [RpcServer.listener,port=50150] ipc.RpcServer: RpcServer.listener,port=50150: stopping 2015-10-14 18:43:00,790 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50150 2015-10-14 18:43:00,790 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:00,790 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:00,791 WARN [Thread-39] ipc.IntegrationTestRpcClient: com.google.protobuf.ServiceException: java.io.IOException: Protocol wrong type for socket 2015-10-14 18:43:01,040 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50201 2015-10-14 18:43:01,041 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:01,041 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50201 2015-10-14 18:43:01,041 INFO [RpcServer.listener,port=50201] ipc.RpcServer: RpcServer.listener,port=50201: starting 2015-10-14 18:43:01,161 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50202 2015-10-14 18:43:01,162 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:01,163 INFO [RpcServer.listener,port=50202] ipc.RpcServer: RpcServer.listener,port=50202: starting 2015-10-14 18:43:01,163 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50202 2015-10-14 18:43:01,279 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50203 2015-10-14 18:43:01,281 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:01,281 INFO [RpcServer.listener,port=50203] ipc.RpcServer: RpcServer.listener,port=50203: starting 2015-10-14 18:43:01,284 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50203 2015-10-14 18:43:01,394 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50202 2015-10-14 18:43:01,394 INFO [Thread-43] ipc.RpcServer: Stopping server on 50202 2015-10-14 18:43:01,395 INFO [RpcServer.listener,port=50202] ipc.RpcServer: RpcServer.listener,port=50202: stopping 2015-10-14 18:43:01,398 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:01,398 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50202 2015-10-14 18:43:01,398 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:01,517 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50206 2015-10-14 18:43:01,519 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:01,519 INFO [RpcServer.listener,port=50206] ipc.RpcServer: RpcServer.listener,port=50206: starting 2015-10-14 18:43:01,519 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50206 2015-10-14 18:43:01,650 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50208 2015-10-14 18:43:01,652 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:01,652 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50208 2015-10-14 18:43:01,652 INFO [RpcServer.listener,port=50208] ipc.RpcServer: RpcServer.listener,port=50208: starting 2015-10-14 18:43:01,766 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50210 2015-10-14 18:43:01,768 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:01,768 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50210 2015-10-14 18:43:01,768 INFO [RpcServer.listener,port=50210] ipc.RpcServer: RpcServer.listener,port=50210: starting 2015-10-14 18:43:01,882 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50211 2015-10-14 18:43:01,883 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:01,883 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50211 2015-10-14 18:43:01,884 INFO [RpcServer.listener,port=50211] ipc.RpcServer: RpcServer.listener,port=50211: starting 2015-10-14 18:43:01,990 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50156 2015-10-14 18:43:01,990 INFO [Thread-43] ipc.RpcServer: Stopping server on 50156 2015-10-14 18:43:01,990 INFO [RpcServer.listener,port=50156] ipc.RpcServer: RpcServer.listener,port=50156: stopping 2015-10-14 18:43:01,991 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:01,991 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:02,009 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50156 2015-10-14 18:43:02,112 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50214 2015-10-14 18:43:02,114 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:02,114 INFO [RpcServer.listener,port=50214] ipc.RpcServer: RpcServer.listener,port=50214: starting 2015-10-14 18:43:02,114 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50214 2015-10-14 18:43:02,233 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50216 2015-10-14 18:43:02,236 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:02,236 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50216 2015-10-14 18:43:02,236 INFO [RpcServer.listener,port=50216] ipc.RpcServer: RpcServer.listener,port=50216: starting 2015-10-14 18:43:02,341 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50146 2015-10-14 18:43:02,341 INFO [Thread-43] ipc.RpcServer: Stopping server on 50146 2015-10-14 18:43:02,341 INFO [RpcServer.listener,port=50146] ipc.RpcServer: RpcServer.listener,port=50146: stopping 2015-10-14 18:43:02,353 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:02,354 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:02,354 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50146 2015-10-14 18:43:02,381 WARN [Thread-17] ipc.IntegrationTestRpcClient: com.google.protobuf.ServiceException: java.io.IOException: Protocol wrong type for socket 2015-10-14 18:43:02,772 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50218 2015-10-14 18:43:02,775 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:02,775 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50218 2015-10-14 18:43:02,775 INFO [RpcServer.listener,port=50218] ipc.RpcServer: RpcServer.listener,port=50218: starting 2015-10-14 18:43:02,991 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50220 2015-10-14 18:43:03,002 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:03,002 INFO [RpcServer.listener,port=50220] ipc.RpcServer: RpcServer.listener,port=50220: starting 2015-10-14 18:43:03,002 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50220 2015-10-14 18:43:03,259 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50222 2015-10-14 18:43:03,261 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:03,261 INFO [RpcServer.listener,port=50222] ipc.RpcServer: RpcServer.listener,port=50222: starting 2015-10-14 18:43:03,261 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50222 2015-10-14 18:43:03,468 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50222 2015-10-14 18:43:03,468 INFO [Thread-43] ipc.RpcServer: Stopping server on 50222 2015-10-14 18:43:03,468 INFO [RpcServer.listener,port=50222] ipc.RpcServer: RpcServer.listener,port=50222: stopping 2015-10-14 18:43:03,475 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50222 2015-10-14 18:43:03,475 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:03,475 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:03,578 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50224 2015-10-14 18:43:03,578 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:03,578 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50224 2015-10-14 18:43:03,578 INFO [RpcServer.listener,port=50224] ipc.RpcServer: RpcServer.listener,port=50224: starting 2015-10-14 18:43:03,700 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50225 2015-10-14 18:43:03,702 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:03,702 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50225 2015-10-14 18:43:03,702 INFO [RpcServer.listener,port=50225] ipc.RpcServer: RpcServer.listener,port=50225: starting 2015-10-14 18:43:03,823 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50227 2015-10-14 18:43:03,826 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:03,826 INFO [RpcServer.listener,port=50227] ipc.RpcServer: RpcServer.listener,port=50227: starting 2015-10-14 18:43:03,826 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50227 2015-10-14 18:43:03,992 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50230 2015-10-14 18:43:03,996 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:04,008 INFO [RpcServer.listener,port=50230] ipc.RpcServer: RpcServer.listener,port=50230: starting 2015-10-14 18:43:04,008 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50230 2015-10-14 18:43:04,168 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50232 2015-10-14 18:43:04,170 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:04,170 INFO [RpcServer.listener,port=50232] ipc.RpcServer: RpcServer.listener,port=50232: starting 2015-10-14 18:43:04,170 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50232 2015-10-14 18:43:04,285 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50234 2015-10-14 18:43:04,287 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:04,288 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50234 2015-10-14 18:43:04,288 INFO [RpcServer.listener,port=50234] ipc.RpcServer: RpcServer.listener,port=50234: starting 2015-10-14 18:43:04,392 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50218 2015-10-14 18:43:04,392 INFO [Thread-43] ipc.RpcServer: Stopping server on 50218 2015-10-14 18:43:04,392 INFO [RpcServer.listener,port=50218] ipc.RpcServer: RpcServer.listener,port=50218: stopping 2015-10-14 18:43:04,394 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50218 2015-10-14 18:43:04,394 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:04,394 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:04,508 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50236 2015-10-14 18:43:04,511 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50236 2015-10-14 18:43:04,511 INFO [RpcServer.listener,port=50236] ipc.RpcServer: RpcServer.listener,port=50236: starting 2015-10-14 18:43:04,511 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:04,731 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50238 2015-10-14 18:43:04,733 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:04,734 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50238 2015-10-14 18:43:04,734 INFO [RpcServer.listener,port=50238] ipc.RpcServer: RpcServer.listener,port=50238: starting 2015-10-14 18:43:04,851 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50239 2015-10-14 18:43:04,853 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:04,853 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50239 2015-10-14 18:43:04,853 INFO [RpcServer.listener,port=50239] ipc.RpcServer: RpcServer.listener,port=50239: starting 2015-10-14 18:43:04,972 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50241 2015-10-14 18:43:04,975 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:04,975 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50241 2015-10-14 18:43:04,975 INFO [RpcServer.listener,port=50241] ipc.RpcServer: RpcServer.listener,port=50241: starting 2015-10-14 18:43:05,125 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50243 2015-10-14 18:43:05,127 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:05,127 INFO [RpcServer.listener,port=50243] ipc.RpcServer: RpcServer.listener,port=50243: starting 2015-10-14 18:43:05,127 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50243 2015-10-14 18:43:05,271 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50246 2015-10-14 18:43:05,274 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:05,274 INFO [RpcServer.listener,port=50246] ipc.RpcServer: RpcServer.listener,port=50246: starting 2015-10-14 18:43:05,274 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50246 2015-10-14 18:43:05,422 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50234 2015-10-14 18:43:05,422 INFO [Thread-43] ipc.RpcServer: Stopping server on 50234 2015-10-14 18:43:05,422 INFO [RpcServer.listener,port=50234] ipc.RpcServer: RpcServer.listener,port=50234: stopping 2015-10-14 18:43:05,436 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:05,436 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50234 2015-10-14 18:43:05,436 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:05,866 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50248 2015-10-14 18:43:05,869 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:05,869 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50248 2015-10-14 18:43:05,869 INFO [RpcServer.listener,port=50248] ipc.RpcServer: RpcServer.listener,port=50248: starting 2015-10-14 18:43:05,973 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50236 2015-10-14 18:43:05,973 INFO [Thread-43] ipc.RpcServer: Stopping server on 50236 2015-10-14 18:43:05,973 INFO [RpcServer.listener,port=50236] ipc.RpcServer: RpcServer.listener,port=50236: stopping 2015-10-14 18:43:05,976 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:05,976 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:05,978 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50236 2015-10-14 18:43:06,099 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50250 2015-10-14 18:43:06,101 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:06,101 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50250 2015-10-14 18:43:06,101 INFO [RpcServer.listener,port=50250] ipc.RpcServer: RpcServer.listener,port=50250: starting 2015-10-14 18:43:06,207 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50241 2015-10-14 18:43:06,207 INFO [Thread-43] ipc.RpcServer: Stopping server on 50241 2015-10-14 18:43:06,209 INFO [RpcServer.listener,port=50241] ipc.RpcServer: RpcServer.listener,port=50241: stopping 2015-10-14 18:43:06,212 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50241 2015-10-14 18:43:06,218 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:06,218 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:06,337 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50252 2015-10-14 18:43:06,339 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:06,339 INFO [RpcServer.listener,port=50252] ipc.RpcServer: RpcServer.listener,port=50252: starting 2015-10-14 18:43:06,339 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50252 2015-10-14 18:43:06,463 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50254 2015-10-14 18:43:06,464 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:06,464 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50254 2015-10-14 18:43:06,464 INFO [RpcServer.listener,port=50254] ipc.RpcServer: RpcServer.listener,port=50254: starting 2015-10-14 18:43:06,571 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50238 2015-10-14 18:43:06,571 INFO [Thread-43] ipc.RpcServer: Stopping server on 50238 2015-10-14 18:43:06,571 INFO [RpcServer.listener,port=50238] ipc.RpcServer: RpcServer.listener,port=50238: stopping 2015-10-14 18:43:06,591 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:06,591 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:06,592 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50238 2015-10-14 18:43:06,696 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50182 2015-10-14 18:43:06,696 INFO [Thread-43] ipc.RpcServer: Stopping server on 50182 2015-10-14 18:43:06,696 INFO [RpcServer.listener,port=50182] ipc.RpcServer: RpcServer.listener,port=50182: stopping 2015-10-14 18:43:06,704 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:06,704 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50182 2015-10-14 18:43:06,705 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:06,809 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50195 2015-10-14 18:43:06,809 INFO [Thread-43] ipc.RpcServer: Stopping server on 50195 2015-10-14 18:43:06,809 INFO [RpcServer.listener,port=50195] ipc.RpcServer: RpcServer.listener,port=50195: stopping 2015-10-14 18:43:06,820 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:06,822 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50195 2015-10-14 18:43:06,822 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:06,928 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50224 2015-10-14 18:43:06,928 INFO [Thread-43] ipc.RpcServer: Stopping server on 50224 2015-10-14 18:43:06,928 INFO [RpcServer.listener,port=50224] ipc.RpcServer: RpcServer.listener,port=50224: stopping 2015-10-14 18:43:06,931 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50224 2015-10-14 18:43:06,931 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:06,931 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:07,039 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50256 2015-10-14 18:43:07,041 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:07,041 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50256 2015-10-14 18:43:07,041 INFO [RpcServer.listener,port=50256] ipc.RpcServer: RpcServer.listener,port=50256: starting 2015-10-14 18:43:07,158 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50257 2015-10-14 18:43:07,160 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:07,160 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50257 2015-10-14 18:43:07,160 INFO [RpcServer.listener,port=50257] ipc.RpcServer: RpcServer.listener,port=50257: starting 2015-10-14 18:43:07,275 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50259 2015-10-14 18:43:07,277 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:07,277 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50259 2015-10-14 18:43:07,277 INFO [RpcServer.listener,port=50259] ipc.RpcServer: RpcServer.listener,port=50259: starting 2015-10-14 18:43:07,512 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50262 2015-10-14 18:43:07,515 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:07,515 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50262 2015-10-14 18:43:07,515 INFO [RpcServer.listener,port=50262] ipc.RpcServer: RpcServer.listener,port=50262: starting 2015-10-14 18:43:07,615 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50176 2015-10-14 18:43:07,615 INFO [Thread-43] ipc.RpcServer: Stopping server on 50176 2015-10-14 18:43:07,615 INFO [RpcServer.listener,port=50176] ipc.RpcServer: RpcServer.listener,port=50176: stopping 2015-10-14 18:43:07,637 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:07,637 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:07,637 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50176 2015-10-14 18:43:07,750 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50264 2015-10-14 18:43:07,752 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:07,752 INFO [RpcServer.listener,port=50264] ipc.RpcServer: RpcServer.listener,port=50264: starting 2015-10-14 18:43:07,752 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50264 2015-10-14 18:43:07,870 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50265 2015-10-14 18:43:07,872 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:07,872 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50265 2015-10-14 18:43:07,872 INFO [RpcServer.listener,port=50265] ipc.RpcServer: RpcServer.listener,port=50265: starting 2015-10-14 18:43:07,985 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50267 2015-10-14 18:43:07,987 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:07,987 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50267 2015-10-14 18:43:07,987 INFO [RpcServer.listener,port=50267] ipc.RpcServer: RpcServer.listener,port=50267: starting 2015-10-14 18:43:08,296 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50216 2015-10-14 18:43:08,297 INFO [Thread-43] ipc.RpcServer: Stopping server on 50216 2015-10-14 18:43:08,297 INFO [RpcServer.listener,port=50216] ipc.RpcServer: RpcServer.listener,port=50216: stopping 2015-10-14 18:43:08,305 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:08,305 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:08,305 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50216 2015-10-14 18:43:08,421 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50270 2015-10-14 18:43:08,425 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:08,425 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50270 2015-10-14 18:43:08,425 INFO [RpcServer.listener,port=50270] ipc.RpcServer: RpcServer.listener,port=50270: starting 2015-10-14 18:43:08,541 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50272 2015-10-14 18:43:08,549 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:08,549 INFO [RpcServer.listener,port=50272] ipc.RpcServer: RpcServer.listener,port=50272: starting 2015-10-14 18:43:08,549 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50272 2015-10-14 18:43:08,663 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50273 2015-10-14 18:43:08,666 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:08,666 INFO [RpcServer.listener,port=50273] ipc.RpcServer: RpcServer.listener,port=50273: starting 2015-10-14 18:43:08,666 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50273 2015-10-14 18:43:08,903 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50274 2015-10-14 18:43:08,906 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:08,906 INFO [RpcServer.listener,port=50274] ipc.RpcServer: RpcServer.listener,port=50274: starting 2015-10-14 18:43:08,913 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50274 2015-10-14 18:43:09,082 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50277 2015-10-14 18:43:09,092 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:09,097 INFO [RpcServer.listener,port=50277] ipc.RpcServer: RpcServer.listener,port=50277: starting 2015-10-14 18:43:09,097 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50277 2015-10-14 18:43:09,216 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50279 2015-10-14 18:43:09,218 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:09,218 INFO [RpcServer.listener,port=50279] ipc.RpcServer: RpcServer.listener,port=50279: starting 2015-10-14 18:43:09,218 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50279 2015-10-14 18:43:09,472 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50282 2015-10-14 18:43:09,474 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:09,474 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50282 2015-10-14 18:43:09,475 INFO [RpcServer.listener,port=50282] ipc.RpcServer: RpcServer.listener,port=50282: starting 2015-10-14 18:43:09,579 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50230 2015-10-14 18:43:09,579 INFO [Thread-43] ipc.RpcServer: Stopping server on 50230 2015-10-14 18:43:09,579 INFO [RpcServer.listener,port=50230] ipc.RpcServer: RpcServer.listener,port=50230: stopping 2015-10-14 18:43:09,581 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50230 2015-10-14 18:43:09,581 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:09,582 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:09,709 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50287 2015-10-14 18:43:09,710 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:09,710 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50287 2015-10-14 18:43:09,710 INFO [RpcServer.listener,port=50287] ipc.RpcServer: RpcServer.listener,port=50287: starting 2015-10-14 18:43:09,811 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50193 2015-10-14 18:43:09,812 INFO [Thread-43] ipc.RpcServer: Stopping server on 50193 2015-10-14 18:43:09,812 INFO [RpcServer.listener,port=50193] ipc.RpcServer: RpcServer.listener,port=50193: stopping 2015-10-14 18:43:09,823 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50193 2015-10-14 18:43:09,823 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:09,823 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:10,046 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50289 2015-10-14 18:43:10,048 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:10,049 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50289 2015-10-14 18:43:10,049 INFO [RpcServer.listener,port=50289] ipc.RpcServer: RpcServer.listener,port=50289: starting 2015-10-14 18:43:10,351 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50259 2015-10-14 18:43:10,351 INFO [Thread-43] ipc.RpcServer: Stopping server on 50259 2015-10-14 18:43:10,352 INFO [RpcServer.listener,port=50259] ipc.RpcServer: RpcServer.listener,port=50259: stopping 2015-10-14 18:43:10,363 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50259 2015-10-14 18:43:10,364 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:10,364 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:10,568 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50277 2015-10-14 18:43:10,568 INFO [Thread-43] ipc.RpcServer: Stopping server on 50277 2015-10-14 18:43:10,568 INFO [RpcServer.listener,port=50277] ipc.RpcServer: RpcServer.listener,port=50277: stopping 2015-10-14 18:43:10,578 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50277 2015-10-14 18:43:10,578 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:10,588 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:10,701 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50292 2015-10-14 18:43:10,703 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:10,704 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50292 2015-10-14 18:43:10,704 INFO [RpcServer.listener,port=50292] ipc.RpcServer: RpcServer.listener,port=50292: starting 2015-10-14 18:43:10,921 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50294 2015-10-14 18:43:10,929 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:10,929 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50294 2015-10-14 18:43:10,929 INFO [RpcServer.listener,port=50294] ipc.RpcServer: RpcServer.listener,port=50294: starting 2015-10-14 18:43:11,078 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50297 2015-10-14 18:43:11,080 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:11,080 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50297 2015-10-14 18:43:11,080 INFO [RpcServer.listener,port=50297] ipc.RpcServer: RpcServer.listener,port=50297: starting 2015-10-14 18:43:11,236 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50300 2015-10-14 18:43:11,246 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:11,246 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50300 2015-10-14 18:43:11,250 INFO [RpcServer.listener,port=50300] ipc.RpcServer: RpcServer.listener,port=50300: starting 2015-10-14 18:43:11,388 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50302 2015-10-14 18:43:11,391 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:11,391 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50302 2015-10-14 18:43:11,391 INFO [RpcServer.listener,port=50302] ipc.RpcServer: RpcServer.listener,port=50302: starting 2015-10-14 18:43:11,518 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50303 2015-10-14 18:43:11,521 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:11,521 INFO [RpcServer.listener,port=50303] ipc.RpcServer: RpcServer.listener,port=50303: starting 2015-10-14 18:43:11,521 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50303 2015-10-14 18:43:11,677 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50306 2015-10-14 18:43:11,680 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:11,680 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50306 2015-10-14 18:43:11,680 INFO [RpcServer.listener,port=50306] ipc.RpcServer: RpcServer.listener,port=50306: starting 2015-10-14 18:43:11,901 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50292 2015-10-14 18:43:11,901 INFO [Thread-43] ipc.RpcServer: Stopping server on 50292 2015-10-14 18:43:11,902 INFO [RpcServer.listener,port=50292] ipc.RpcServer: RpcServer.listener,port=50292: stopping 2015-10-14 18:43:11,913 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50292 2015-10-14 18:43:11,913 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:11,913 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:12,015 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50187 2015-10-14 18:43:12,015 INFO [Thread-43] ipc.RpcServer: Stopping server on 50187 2015-10-14 18:43:12,015 INFO [RpcServer.listener,port=50187] ipc.RpcServer: RpcServer.listener,port=50187: stopping 2015-10-14 18:43:12,023 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50187 2015-10-14 18:43:12,023 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:12,023 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:12,127 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50297 2015-10-14 18:43:12,127 INFO [Thread-43] ipc.RpcServer: Stopping server on 50297 2015-10-14 18:43:12,128 INFO [RpcServer.listener,port=50297] ipc.RpcServer: RpcServer.listener,port=50297: stopping 2015-10-14 18:43:12,130 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50297 2015-10-14 18:43:12,130 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:12,130 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:12,233 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50210 2015-10-14 18:43:12,233 INFO [Thread-43] ipc.RpcServer: Stopping server on 50210 2015-10-14 18:43:12,233 INFO [RpcServer.listener,port=50210] ipc.RpcServer: RpcServer.listener,port=50210: stopping 2015-10-14 18:43:12,240 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50210 2015-10-14 18:43:12,240 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:12,240 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:12,450 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50308 2015-10-14 18:43:12,458 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:12,458 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50308 2015-10-14 18:43:12,458 INFO [RpcServer.listener,port=50308] ipc.RpcServer: RpcServer.listener,port=50308: starting 2015-10-14 18:43:12,563 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50147 2015-10-14 18:43:12,563 INFO [Thread-43] ipc.RpcServer: Stopping server on 50147 2015-10-14 18:43:12,563 INFO [RpcServer.listener,port=50147] ipc.RpcServer: RpcServer.listener,port=50147: stopping 2015-10-14 18:43:12,574 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50147 2015-10-14 18:43:12,574 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:12,574 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:12,689 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50316 2015-10-14 18:43:12,697 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:12,698 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50316 2015-10-14 18:43:12,699 INFO [RpcServer.listener,port=50316] ipc.RpcServer: RpcServer.listener,port=50316: starting 2015-10-14 18:43:12,810 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50155 2015-10-14 18:43:12,810 INFO [Thread-43] ipc.RpcServer: Stopping server on 50155 2015-10-14 18:43:12,810 INFO [RpcServer.listener,port=50155] ipc.RpcServer: RpcServer.listener,port=50155: stopping 2015-10-14 18:43:12,821 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50155 2015-10-14 18:43:12,821 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:12,821 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:12,925 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50316 2015-10-14 18:43:12,925 INFO [Thread-43] ipc.RpcServer: Stopping server on 50316 2015-10-14 18:43:12,925 INFO [RpcServer.listener,port=50316] ipc.RpcServer: RpcServer.listener,port=50316: stopping 2015-10-14 18:43:12,938 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50316 2015-10-14 18:43:12,939 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:12,939 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:13,044 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50318 2015-10-14 18:43:13,045 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:13,045 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50318 2015-10-14 18:43:13,045 INFO [RpcServer.listener,port=50318] ipc.RpcServer: RpcServer.listener,port=50318: starting 2015-10-14 18:43:13,259 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50320 2015-10-14 18:43:13,260 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50320 2015-10-14 18:43:13,260 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:13,260 INFO [RpcServer.listener,port=50320] ipc.RpcServer: RpcServer.listener,port=50320: starting 2015-10-14 18:43:13,365 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50220 2015-10-14 18:43:13,365 INFO [Thread-43] ipc.RpcServer: Stopping server on 50220 2015-10-14 18:43:13,365 INFO [RpcServer.listener,port=50220] ipc.RpcServer: RpcServer.listener,port=50220: stopping 2015-10-14 18:43:13,377 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50220 2015-10-14 18:43:13,384 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:13,384 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:13,491 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50320 2015-10-14 18:43:13,492 INFO [Thread-43] ipc.RpcServer: Stopping server on 50320 2015-10-14 18:43:13,492 INFO [RpcServer.listener,port=50320] ipc.RpcServer: RpcServer.listener,port=50320: stopping 2015-10-14 18:43:13,498 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50320 2015-10-14 18:43:13,498 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:13,498 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:13,908 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50211 2015-10-14 18:43:13,908 INFO [Thread-43] ipc.RpcServer: Stopping server on 50211 2015-10-14 18:43:13,908 INFO [RpcServer.listener,port=50211] ipc.RpcServer: RpcServer.listener,port=50211: stopping 2015-10-14 18:43:13,918 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50211 2015-10-14 18:43:13,918 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:13,918 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:13,933 WARN [FifoRpcScheduler.handler35-thread-1] ipc.RpcServer: FifoRpcScheduler.handler35-thread-1: caught a ClosedChannelException, this means that the server /127.0.0.1:50211 was processing a request but the client went away. The error message was: null 2015-10-14 18:43:14,037 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50323 2015-10-14 18:43:14,040 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:14,040 INFO [RpcServer.listener,port=50323] ipc.RpcServer: RpcServer.listener,port=50323: starting 2015-10-14 18:43:14,040 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50323 2015-10-14 18:43:14,246 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50203 2015-10-14 18:43:14,247 INFO [Thread-43] ipc.RpcServer: Stopping server on 50203 2015-10-14 18:43:14,247 INFO [RpcServer.listener,port=50203] ipc.RpcServer: RpcServer.listener,port=50203: stopping 2015-10-14 18:43:14,252 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50203 2015-10-14 18:43:14,252 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:14,252 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:14,358 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50325 2015-10-14 18:43:14,360 INFO [RpcServer.listener,port=50325] ipc.RpcServer: RpcServer.listener,port=50325: starting 2015-10-14 18:43:14,360 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50325 2015-10-14 18:43:14,360 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:14,578 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50270 2015-10-14 18:43:14,578 INFO [Thread-43] ipc.RpcServer: Stopping server on 50270 2015-10-14 18:43:14,578 INFO [RpcServer.listener,port=50270] ipc.RpcServer: RpcServer.listener,port=50270: stopping 2015-10-14 18:43:14,580 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:14,580 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:14,580 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50270 2015-10-14 18:43:14,682 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50227 2015-10-14 18:43:14,682 INFO [Thread-43] ipc.RpcServer: Stopping server on 50227 2015-10-14 18:43:14,683 INFO [RpcServer.listener,port=50227] ipc.RpcServer: RpcServer.listener,port=50227: stopping 2015-10-14 18:43:14,687 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50227 2015-10-14 18:43:14,687 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:14,687 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:14,810 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50329 2015-10-14 18:43:14,812 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:14,812 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50329 2015-10-14 18:43:14,812 INFO [RpcServer.listener,port=50329] ipc.RpcServer: RpcServer.listener,port=50329: starting 2015-10-14 18:43:14,934 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50330 2015-10-14 18:43:14,937 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:14,937 INFO [RpcServer.listener,port=50330] ipc.RpcServer: RpcServer.listener,port=50330: starting 2015-10-14 18:43:14,937 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50330 2015-10-14 18:43:15,255 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50335 2015-10-14 18:43:15,257 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:15,257 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50335 2015-10-14 18:43:15,257 INFO [RpcServer.listener,port=50335] ipc.RpcServer: RpcServer.listener,port=50335: starting 2015-10-14 18:43:15,377 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50337 2015-10-14 18:43:15,379 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50337 2015-10-14 18:43:15,379 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:15,379 INFO [RpcServer.listener,port=50337] ipc.RpcServer: RpcServer.listener,port=50337: starting 2015-10-14 18:43:15,499 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50340 2015-10-14 18:43:15,502 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:15,502 INFO [RpcServer.listener,port=50340] ipc.RpcServer: RpcServer.listener,port=50340: starting 2015-10-14 18:43:15,502 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50340 2015-10-14 18:43:15,710 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50274 2015-10-14 18:43:15,710 INFO [Thread-43] ipc.RpcServer: Stopping server on 50274 2015-10-14 18:43:15,710 INFO [RpcServer.listener,port=50274] ipc.RpcServer: RpcServer.listener,port=50274: stopping 2015-10-14 18:43:15,712 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50274 2015-10-14 18:43:15,712 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:15,712 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:15,837 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50344 2015-10-14 18:43:15,840 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:15,840 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50344 2015-10-14 18:43:15,840 INFO [RpcServer.listener,port=50344] ipc.RpcServer: RpcServer.listener,port=50344: starting 2015-10-14 18:43:15,960 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50345 2015-10-14 18:43:15,963 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:15,968 INFO [RpcServer.listener,port=50345] ipc.RpcServer: RpcServer.listener,port=50345: starting 2015-10-14 18:43:15,976 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50345 2015-10-14 18:43:16,100 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50346 2015-10-14 18:43:16,110 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:16,110 INFO [RpcServer.listener,port=50346] ipc.RpcServer: RpcServer.listener,port=50346: starting 2015-10-14 18:43:16,110 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50346 2015-10-14 18:43:16,243 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50347 2015-10-14 18:43:16,247 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:16,247 INFO [RpcServer.listener,port=50347] ipc.RpcServer: RpcServer.listener,port=50347: starting 2015-10-14 18:43:16,247 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50347 2015-10-14 18:43:16,349 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50289 2015-10-14 18:43:16,349 INFO [Thread-43] ipc.RpcServer: Stopping server on 50289 2015-10-14 18:43:16,350 INFO [RpcServer.listener,port=50289] ipc.RpcServer: RpcServer.listener,port=50289: stopping 2015-10-14 18:43:16,352 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50289 2015-10-14 18:43:16,353 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:16,353 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:16,454 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50225 2015-10-14 18:43:16,454 INFO [Thread-43] ipc.RpcServer: Stopping server on 50225 2015-10-14 18:43:16,454 INFO [RpcServer.listener,port=50225] ipc.RpcServer: RpcServer.listener,port=50225: stopping 2015-10-14 18:43:16,455 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50225 2015-10-14 18:43:16,456 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:16,457 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:16,557 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50250 2015-10-14 18:43:16,557 INFO [Thread-43] ipc.RpcServer: Stopping server on 50250 2015-10-14 18:43:16,559 INFO [RpcServer.listener,port=50250] ipc.RpcServer: RpcServer.listener,port=50250: stopping 2015-10-14 18:43:16,561 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50250 2015-10-14 18:43:16,562 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:16,563 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:16,772 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50352 2015-10-14 18:43:16,774 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:16,774 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50352 2015-10-14 18:43:16,774 INFO [RpcServer.listener,port=50352] ipc.RpcServer: RpcServer.listener,port=50352: starting 2015-10-14 18:43:16,996 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50355 2015-10-14 18:43:16,998 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50355 2015-10-14 18:43:16,998 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:16,998 INFO [RpcServer.listener,port=50355] ipc.RpcServer: RpcServer.listener,port=50355: starting 2015-10-14 18:43:17,328 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50358 2015-10-14 18:43:17,336 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:17,336 INFO [RpcServer.listener,port=50358] ipc.RpcServer: RpcServer.listener,port=50358: starting 2015-10-14 18:43:17,337 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50358 2015-10-14 18:43:17,452 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50359 2015-10-14 18:43:17,454 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:17,455 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50359 2015-10-14 18:43:17,456 INFO [RpcServer.listener,port=50359] ipc.RpcServer: RpcServer.listener,port=50359: starting 2015-10-14 18:43:17,577 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50361 2015-10-14 18:43:17,590 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:17,591 INFO [RpcServer.listener,port=50361] ipc.RpcServer: RpcServer.listener,port=50361: starting 2015-10-14 18:43:17,591 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50361 2015-10-14 18:43:17,696 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50246 2015-10-14 18:43:17,698 INFO [Thread-43] ipc.RpcServer: Stopping server on 50246 2015-10-14 18:43:17,698 INFO [RpcServer.listener,port=50246] ipc.RpcServer: RpcServer.listener,port=50246: stopping 2015-10-14 18:43:17,706 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:17,706 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:17,706 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50246 2015-10-14 18:43:17,841 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50364 2015-10-14 18:43:17,844 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:17,844 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50364 2015-10-14 18:43:17,844 INFO [RpcServer.listener,port=50364] ipc.RpcServer: RpcServer.listener,port=50364: starting 2015-10-14 18:43:18,070 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50368 2015-10-14 18:43:18,072 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:18,073 INFO [RpcServer.listener,port=50368] ipc.RpcServer: RpcServer.listener,port=50368: starting 2015-10-14 18:43:18,073 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50368 2015-10-14 18:43:18,207 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50369 2015-10-14 18:43:18,210 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:18,210 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50369 2015-10-14 18:43:18,210 INFO [RpcServer.listener,port=50369] ipc.RpcServer: RpcServer.listener,port=50369: starting 2015-10-14 18:43:18,333 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50370 2015-10-14 18:43:18,335 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:18,336 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50370 2015-10-14 18:43:18,336 INFO [RpcServer.listener,port=50370] ipc.RpcServer: RpcServer.listener,port=50370: starting 2015-10-14 18:43:18,442 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50308 2015-10-14 18:43:18,442 INFO [Thread-43] ipc.RpcServer: Stopping server on 50308 2015-10-14 18:43:18,443 INFO [RpcServer.listener,port=50308] ipc.RpcServer: RpcServer.listener,port=50308: stopping 2015-10-14 18:43:18,460 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50308 2015-10-14 18:43:18,460 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:18,461 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:18,561 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50318 2015-10-14 18:43:18,562 INFO [Thread-43] ipc.RpcServer: Stopping server on 50318 2015-10-14 18:43:18,563 INFO [RpcServer.listener,port=50318] ipc.RpcServer: RpcServer.listener,port=50318: stopping 2015-10-14 18:43:18,589 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50318 2015-10-14 18:43:18,589 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:18,589 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:18,713 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50374 2015-10-14 18:43:18,715 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:18,715 INFO [RpcServer.listener,port=50374] ipc.RpcServer: RpcServer.listener,port=50374: starting 2015-10-14 18:43:18,715 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50374 2015-10-14 18:43:18,842 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50375 2015-10-14 18:43:18,844 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:18,844 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50375 2015-10-14 18:43:18,844 INFO [RpcServer.listener,port=50375] ipc.RpcServer: RpcServer.listener,port=50375: starting 2015-10-14 18:43:18,972 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50376 2015-10-14 18:43:18,975 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:18,975 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50376 2015-10-14 18:43:18,975 INFO [RpcServer.listener,port=50376] ipc.RpcServer: RpcServer.listener,port=50376: starting 2015-10-14 18:43:19,084 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50359 2015-10-14 18:43:19,084 INFO [Thread-43] ipc.RpcServer: Stopping server on 50359 2015-10-14 18:43:19,085 INFO [RpcServer.listener,port=50359] ipc.RpcServer: RpcServer.listener,port=50359: stopping 2015-10-14 18:43:19,086 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50359 2015-10-14 18:43:19,087 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:19,087 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:19,187 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50191 2015-10-14 18:43:19,187 INFO [Thread-43] ipc.RpcServer: Stopping server on 50191 2015-10-14 18:43:19,188 INFO [RpcServer.listener,port=50191] ipc.RpcServer: RpcServer.listener,port=50191: stopping 2015-10-14 18:43:19,200 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:19,200 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:19,219 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50191 2015-10-14 18:43:19,327 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50380 2015-10-14 18:43:19,328 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:19,328 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50380 2015-10-14 18:43:19,328 INFO [RpcServer.listener,port=50380] ipc.RpcServer: RpcServer.listener,port=50380: starting 2015-10-14 18:43:19,450 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50382 2015-10-14 18:43:19,453 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:19,453 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50382 2015-10-14 18:43:19,453 INFO [RpcServer.listener,port=50382] ipc.RpcServer: RpcServer.listener,port=50382: starting 2015-10-14 18:43:19,593 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50384 2015-10-14 18:43:19,596 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:19,597 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50384 2015-10-14 18:43:19,609 INFO [RpcServer.listener,port=50384] ipc.RpcServer: RpcServer.listener,port=50384: starting 2015-10-14 18:43:19,697 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50265 2015-10-14 18:43:19,697 INFO [Thread-43] ipc.RpcServer: Stopping server on 50265 2015-10-14 18:43:19,698 INFO [RpcServer.listener,port=50265] ipc.RpcServer: RpcServer.listener,port=50265: stopping 2015-10-14 18:43:19,704 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50265 2015-10-14 18:43:19,704 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:19,704 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:19,830 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50385 2015-10-14 18:43:19,833 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:19,833 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50385 2015-10-14 18:43:19,833 INFO [RpcServer.listener,port=50385] ipc.RpcServer: RpcServer.listener,port=50385: starting 2015-10-14 18:43:19,984 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50387 2015-10-14 18:43:19,992 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:19,993 INFO [RpcServer.listener,port=50387] ipc.RpcServer: RpcServer.listener,port=50387: starting 2015-10-14 18:43:19,993 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50387 2015-10-14 18:43:20,118 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50389 2015-10-14 18:43:20,128 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:20,128 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50389 2015-10-14 18:43:20,128 INFO [RpcServer.listener,port=50389] ipc.RpcServer: RpcServer.listener,port=50389: starting 2015-10-14 18:43:20,246 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50330 2015-10-14 18:43:20,248 INFO [Thread-43] ipc.RpcServer: Stopping server on 50330 2015-10-14 18:43:20,250 INFO [RpcServer.listener,port=50330] ipc.RpcServer: RpcServer.listener,port=50330: stopping 2015-10-14 18:43:20,263 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:20,263 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:20,263 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50330 2015-10-14 18:43:20,367 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50208 2015-10-14 18:43:20,368 INFO [Thread-43] ipc.RpcServer: Stopping server on 50208 2015-10-14 18:43:20,368 INFO [RpcServer.listener,port=50208] ipc.RpcServer: RpcServer.listener,port=50208: stopping 2015-10-14 18:43:20,390 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:20,390 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:20,392 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50208 2015-10-14 18:43:20,495 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50201 2015-10-14 18:43:20,496 INFO [Thread-43] ipc.RpcServer: Stopping server on 50201 2015-10-14 18:43:20,496 INFO [RpcServer.listener,port=50201] ipc.RpcServer: RpcServer.listener,port=50201: stopping 2015-10-14 18:43:20,508 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:20,509 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:20,509 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50201 2015-10-14 18:43:20,624 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50392 2015-10-14 18:43:20,626 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:20,626 INFO [RpcServer.listener,port=50392] ipc.RpcServer: RpcServer.listener,port=50392: starting 2015-10-14 18:43:20,626 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50392 2015-10-14 18:43:20,750 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50393 2015-10-14 18:43:20,762 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:20,764 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50393 2015-10-14 18:43:20,764 INFO [RpcServer.listener,port=50393] ipc.RpcServer: RpcServer.listener,port=50393: starting 2015-10-14 18:43:20,987 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50396 2015-10-14 18:43:20,989 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:20,990 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50396 2015-10-14 18:43:20,990 INFO [RpcServer.listener,port=50396] ipc.RpcServer: RpcServer.listener,port=50396: starting 2015-10-14 18:43:21,196 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50248 2015-10-14 18:43:21,197 INFO [Thread-43] ipc.RpcServer: Stopping server on 50248 2015-10-14 18:43:21,201 INFO [RpcServer.listener,port=50248] ipc.RpcServer: RpcServer.listener,port=50248: stopping 2015-10-14 18:43:21,219 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:21,220 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:21,220 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50248 2015-10-14 18:43:21,649 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50398 2015-10-14 18:43:21,651 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:21,651 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50398 2015-10-14 18:43:21,651 INFO [RpcServer.listener,port=50398] ipc.RpcServer: RpcServer.listener,port=50398: starting 2015-10-14 18:43:21,854 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50306 2015-10-14 18:43:21,854 INFO [Thread-43] ipc.RpcServer: Stopping server on 50306 2015-10-14 18:43:21,854 INFO [RpcServer.listener,port=50306] ipc.RpcServer: RpcServer.listener,port=50306: stopping 2015-10-14 18:43:21,861 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:21,861 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:21,861 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50306 2015-10-14 18:43:22,093 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50400 2015-10-14 18:43:22,096 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:22,096 INFO [RpcServer.listener,port=50400] ipc.RpcServer: RpcServer.listener,port=50400: starting 2015-10-14 18:43:22,096 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50400 2015-10-14 18:43:22,217 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50401 2015-10-14 18:43:22,221 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:22,221 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50401 2015-10-14 18:43:22,221 INFO [RpcServer.listener,port=50401] ipc.RpcServer: RpcServer.listener,port=50401: starting 2015-10-14 18:43:22,340 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50402 2015-10-14 18:43:22,344 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:22,344 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50402 2015-10-14 18:43:22,344 INFO [RpcServer.listener,port=50402] ipc.RpcServer: RpcServer.listener,port=50402: starting 2015-10-14 18:43:22,473 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50404 2015-10-14 18:43:22,476 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:22,476 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50404 2015-10-14 18:43:22,476 INFO [RpcServer.listener,port=50404] ipc.RpcServer: RpcServer.listener,port=50404: starting 2015-10-14 18:43:22,585 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50344 2015-10-14 18:43:22,585 INFO [Thread-43] ipc.RpcServer: Stopping server on 50344 2015-10-14 18:43:22,585 INFO [RpcServer.listener,port=50344] ipc.RpcServer: RpcServer.listener,port=50344: stopping 2015-10-14 18:43:22,598 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:22,598 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:22,600 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50344 2015-10-14 18:43:22,703 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50376 2015-10-14 18:43:22,703 INFO [Thread-43] ipc.RpcServer: Stopping server on 50376 2015-10-14 18:43:22,703 INFO [RpcServer.listener,port=50376] ipc.RpcServer: RpcServer.listener,port=50376: stopping 2015-10-14 18:43:22,715 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:22,715 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:22,715 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50376 2015-10-14 18:43:22,842 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50407 2015-10-14 18:43:22,845 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50407 2015-10-14 18:43:22,845 INFO [RpcServer.listener,port=50407] ipc.RpcServer: RpcServer.listener,port=50407: starting 2015-10-14 18:43:22,845 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:22,962 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50408 2015-10-14 18:43:22,972 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:22,972 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50408 2015-10-14 18:43:22,974 INFO [RpcServer.listener,port=50408] ipc.RpcServer: RpcServer.listener,port=50408: starting 2015-10-14 18:43:23,181 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50267 2015-10-14 18:43:23,182 INFO [Thread-43] ipc.RpcServer: Stopping server on 50267 2015-10-14 18:43:23,183 INFO [RpcServer.listener,port=50267] ipc.RpcServer: RpcServer.listener,port=50267: stopping 2015-10-14 18:43:23,198 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50267 2015-10-14 18:43:23,199 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:23,200 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:23,311 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50396 2015-10-14 18:43:23,311 INFO [Thread-43] ipc.RpcServer: Stopping server on 50396 2015-10-14 18:43:23,315 INFO [RpcServer.listener,port=50396] ipc.RpcServer: RpcServer.listener,port=50396: stopping 2015-10-14 18:43:23,337 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50396 2015-10-14 18:43:23,339 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:23,339 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:23,442 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50272 2015-10-14 18:43:23,442 INFO [Thread-43] ipc.RpcServer: Stopping server on 50272 2015-10-14 18:43:23,442 INFO [RpcServer.listener,port=50272] ipc.RpcServer: RpcServer.listener,port=50272: stopping 2015-10-14 18:43:23,453 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50272 2015-10-14 18:43:23,453 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:23,453 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:23,555 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50262 2015-10-14 18:43:23,555 INFO [Thread-43] ipc.RpcServer: Stopping server on 50262 2015-10-14 18:43:23,555 INFO [RpcServer.listener,port=50262] ipc.RpcServer: RpcServer.listener,port=50262: stopping 2015-10-14 18:43:23,569 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50262 2015-10-14 18:43:23,569 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:23,569 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:23,600 WARN [FifoRpcScheduler.handler60-thread-1] ipc.RpcServer: FifoRpcScheduler.handler60-thread-1: caught a ClosedChannelException, this means that the server /127.0.0.1:50262 was processing a request but the client went away. The error message was: null 2015-10-14 18:43:23,788 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50412 2015-10-14 18:43:23,790 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:23,790 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50412 2015-10-14 18:43:23,790 INFO [RpcServer.listener,port=50412] ipc.RpcServer: RpcServer.listener,port=50412: starting 2015-10-14 18:43:23,905 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50413 2015-10-14 18:43:23,908 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:23,908 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50413 2015-10-14 18:43:23,908 INFO [RpcServer.listener,port=50413] ipc.RpcServer: RpcServer.listener,port=50413: starting 2015-10-14 18:43:24,008 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50287 2015-10-14 18:43:24,009 INFO [Thread-43] ipc.RpcServer: Stopping server on 50287 2015-10-14 18:43:24,009 INFO [RpcServer.listener,port=50287] ipc.RpcServer: RpcServer.listener,port=50287: stopping 2015-10-14 18:43:24,019 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:24,019 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:24,019 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50287 2015-10-14 18:43:24,172 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50414 2015-10-14 18:43:24,176 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:24,177 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50414 2015-10-14 18:43:24,177 INFO [RpcServer.listener,port=50414] ipc.RpcServer: RpcServer.listener,port=50414: starting 2015-10-14 18:43:24,293 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50417 2015-10-14 18:43:24,302 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:24,302 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50417 2015-10-14 18:43:24,302 INFO [RpcServer.listener,port=50417] ipc.RpcServer: RpcServer.listener,port=50417: starting 2015-10-14 18:43:24,429 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50418 2015-10-14 18:43:24,437 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:24,437 INFO [RpcServer.listener,port=50418] ipc.RpcServer: RpcServer.listener,port=50418: starting 2015-10-14 18:43:24,437 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50418 2015-10-14 18:43:24,559 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50420 2015-10-14 18:43:24,565 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:24,565 INFO [RpcServer.listener,port=50420] ipc.RpcServer: RpcServer.listener,port=50420: starting 2015-10-14 18:43:24,565 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50420 2015-10-14 18:43:24,692 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50422 2015-10-14 18:43:24,701 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:24,701 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50422 2015-10-14 18:43:24,725 INFO [RpcServer.listener,port=50422] ipc.RpcServer: RpcServer.listener,port=50422: starting 2015-10-14 18:43:24,807 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50329 2015-10-14 18:43:24,808 INFO [Thread-43] ipc.RpcServer: Stopping server on 50329 2015-10-14 18:43:24,808 INFO [RpcServer.listener,port=50329] ipc.RpcServer: RpcServer.listener,port=50329: stopping 2015-10-14 18:43:24,820 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:24,820 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:24,822 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50329 2015-10-14 18:43:24,926 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50417 2015-10-14 18:43:24,926 INFO [Thread-43] ipc.RpcServer: Stopping server on 50417 2015-10-14 18:43:24,927 INFO [RpcServer.listener,port=50417] ipc.RpcServer: RpcServer.listener,port=50417: stopping 2015-10-14 18:43:24,935 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50417 2015-10-14 18:43:24,935 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:24,936 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:25,051 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50426 2015-10-14 18:43:25,053 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:25,053 INFO [RpcServer.listener,port=50426] ipc.RpcServer: RpcServer.listener,port=50426: starting 2015-10-14 18:43:25,053 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50426 2015-10-14 18:43:25,173 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50427 2015-10-14 18:43:25,176 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50427 2015-10-14 18:43:25,176 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:25,176 INFO [RpcServer.listener,port=50427] ipc.RpcServer: RpcServer.listener,port=50427: starting 2015-10-14 18:43:25,626 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50430 2015-10-14 18:43:25,628 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:25,629 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50430 2015-10-14 18:43:25,629 INFO [RpcServer.listener,port=50430] ipc.RpcServer: RpcServer.listener,port=50430: starting 2015-10-14 18:43:25,757 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50431 2015-10-14 18:43:25,761 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50431 2015-10-14 18:43:25,761 INFO [RpcServer.listener,port=50431] ipc.RpcServer: RpcServer.listener,port=50431: starting 2015-10-14 18:43:25,761 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:25,911 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50433 2015-10-14 18:43:25,912 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50433 2015-10-14 18:43:25,912 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:25,912 INFO [RpcServer.listener,port=50433] ipc.RpcServer: RpcServer.listener,port=50433: starting 2015-10-14 18:43:26,051 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50436 2015-10-14 18:43:26,055 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:26,055 INFO [RpcServer.listener,port=50436] ipc.RpcServer: RpcServer.listener,port=50436: starting 2015-10-14 18:43:26,056 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50436 2015-10-14 18:43:26,156 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopping server: /127.0.0.1:50358 2015-10-14 18:43:26,157 INFO [Thread-43] ipc.RpcServer: Stopping server on 50358 2015-10-14 18:43:26,157 INFO [RpcServer.listener,port=50358] ipc.RpcServer: RpcServer.listener,port=50358: stopping 2015-10-14 18:43:26,161 INFO [Thread-43] ipc.IntegrationTestRpcClient: Stopped server: /127.0.0.1:50358 2015-10-14 18:43:26,161 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2015-10-14 18:43:26,161 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2015-10-14 18:43:26,391 INFO [Thread-43] ipc.RpcServer: testRpcServer: started 10 reader(s) listening on port=50438 2015-10-14 18:43:26,394 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting 2015-10-14 18:43:26,394 INFO [RpcServer.listener,port=50438] ipc.RpcServer: RpcServer.listener,port=50438: starting 2015-10-14 18:43:26,394 INFO [Thread-43] ipc.IntegrationTestRpcClient: Started server: /127.0.0.1:50438 2015-10-14 18:43:26,530 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Stopping MiniChaosMonkey 2015-10-14 18:43:26,602 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Stopping clients 2015-10-14 18:43:26,602 INFO [Time-limited test] ipc.IntegrationTestRpcClient: Stopping client: client_0_ 2015-10-14 18:43:26,602 INFO [Time-limited test] ipc.IntegrationTestRpcClient: client_0_ numCalls:2 Process Thread Dump: TEST TIMEOUT STACK DUMP 950 active threads Thread 1825 (FifoRpcScheduler.handler136-thread-1): State: WAITING Blocked count: 0 Waited count: 196 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1303acc1 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1813 (RpcServer.listener,port=50438): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1824 (RpcServer.responder): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1823 (RpcServer.reader=9,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1822 (RpcServer.reader=8,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1821 (RpcServer.reader=7,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1820 (RpcServer.reader=6,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1819 (RpcServer.reader=5,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1818 (RpcServer.reader=4,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1817 (RpcServer.reader=3,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1816 (RpcServer.reader=2,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1815 (RpcServer.reader=1,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1814 (RpcServer.reader=0,bindAddress=localhost,port=50438): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1812 (FifoRpcScheduler.handler135-thread-1): State: WAITING Blocked count: 0 Waited count: 216 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1439e04b Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1800 (RpcServer.listener,port=50436): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1811 (RpcServer.responder): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1810 (RpcServer.reader=9,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1809 (RpcServer.reader=8,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1808 (RpcServer.reader=7,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1807 (RpcServer.reader=6,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1806 (RpcServer.reader=5,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1805 (RpcServer.reader=4,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1804 (RpcServer.reader=3,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1803 (RpcServer.reader=2,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1802 (RpcServer.reader=1,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1801 (RpcServer.reader=0,bindAddress=localhost,port=50436): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1799 (FifoRpcScheduler.handler134-thread-1): State: WAITING Blocked count: 1 Waited count: 205 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6591cea1 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1798 (FifoRpcScheduler.handler133-thread-1): State: WAITING Blocked count: 0 Waited count: 193 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7f0637ef Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1786 (RpcServer.listener,port=50433): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1797 (RpcServer.responder): State: RUNNABLE Blocked count: 13 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1796 (RpcServer.reader=9,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1795 (RpcServer.reader=8,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1794 (RpcServer.reader=7,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1793 (RpcServer.reader=6,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1792 (RpcServer.reader=5,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1791 (RpcServer.reader=4,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1790 (RpcServer.reader=3,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1789 (RpcServer.reader=2,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1788 (RpcServer.reader=1,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1787 (RpcServer.reader=0,bindAddress=localhost,port=50433): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1785 (FifoRpcScheduler.handler132-thread-1): State: WAITING Blocked count: 1 Waited count: 211 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@233e1589 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1773 (RpcServer.listener,port=50431): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1784 (RpcServer.responder): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1783 (RpcServer.reader=9,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1782 (RpcServer.reader=8,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1781 (RpcServer.reader=7,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1780 (RpcServer.reader=6,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1779 (RpcServer.reader=5,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1778 (RpcServer.reader=4,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1777 (RpcServer.reader=3,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1776 (RpcServer.reader=2,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1775 (RpcServer.reader=1,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1774 (RpcServer.reader=0,bindAddress=localhost,port=50431): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1761 (RpcServer.listener,port=50430): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1772 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1771 (RpcServer.reader=9,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1770 (RpcServer.reader=8,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1769 (RpcServer.reader=7,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1768 (RpcServer.reader=6,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1767 (RpcServer.reader=5,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1766 (RpcServer.reader=4,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1765 (RpcServer.reader=3,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1764 (RpcServer.reader=2,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1763 (RpcServer.reader=1,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1762 (RpcServer.reader=0,bindAddress=localhost,port=50430): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1760 (FifoRpcScheduler.handler131-thread-1): State: WAITING Blocked count: 0 Waited count: 199 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@28666c0 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1759 (FifoRpcScheduler.handler130-thread-1): State: WAITING Blocked count: 1 Waited count: 184 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@661e855f Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1747 (RpcServer.listener,port=50427): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1758 (RpcServer.responder): State: RUNNABLE Blocked count: 6 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1757 (RpcServer.reader=9,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1756 (RpcServer.reader=8,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1755 (RpcServer.reader=7,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1754 (RpcServer.reader=6,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1753 (RpcServer.reader=5,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1752 (RpcServer.reader=4,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1751 (RpcServer.reader=3,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1750 (RpcServer.reader=2,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1749 (RpcServer.reader=1,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1748 (RpcServer.reader=0,bindAddress=localhost,port=50427): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1735 (RpcServer.listener,port=50426): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1746 (RpcServer.responder): State: RUNNABLE Blocked count: 6 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1745 (RpcServer.reader=9,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1744 (RpcServer.reader=8,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1743 (RpcServer.reader=7,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1742 (RpcServer.reader=6,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1741 (RpcServer.reader=5,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1740 (RpcServer.reader=4,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1739 (RpcServer.reader=3,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1738 (RpcServer.reader=2,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1737 (RpcServer.reader=1,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1736 (RpcServer.reader=0,bindAddress=localhost,port=50426): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1734 (FifoRpcScheduler.handler129-thread-1): State: RUNNABLE Blocked count: 0 Waited count: 199 Stack: java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304) java.lang.StringCoding.encode(StringCoding.java:344) java.lang.String.getBytes(String.java:916) com.google.protobuf.ByteString.copyFromUtf8(ByteString.java:255) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto.getMessageBytes(TestProtos.java:1351) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto.getSerializedSize(TestProtos.java:1393) org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2149) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$TestRpcServer.call(IntegrationTestRpcClient.java:98) org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:106) org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73) java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) java.util.concurrent.FutureTask.run(FutureTask.java:262) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1733 (FifoRpcScheduler.handler128-thread-1): State: WAITING Blocked count: 0 Waited count: 230 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5ba0ad71 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1732 (FifoRpcScheduler.handler123-thread-1): State: WAITING Blocked count: 0 Waited count: 222 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2728b119 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1720 (RpcServer.listener,port=50422): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1731 (RpcServer.responder): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1730 (RpcServer.reader=9,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1729 (RpcServer.reader=8,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1728 (RpcServer.reader=7,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1727 (RpcServer.reader=6,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1726 (RpcServer.reader=5,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1725 (RpcServer.reader=4,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1724 (RpcServer.reader=3,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1723 (RpcServer.reader=2,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1722 (RpcServer.reader=1,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1721 (RpcServer.reader=0,bindAddress=localhost,port=50422): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1706 (RpcServer.listener,port=50420): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1718 (RpcServer.responder): State: RUNNABLE Blocked count: 3 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1717 (RpcServer.reader=9,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1716 (RpcServer.reader=8,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1715 (RpcServer.reader=7,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1714 (RpcServer.reader=6,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1713 (RpcServer.reader=5,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1712 (RpcServer.reader=4,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1711 (FifoRpcScheduler.handler127-thread-1): State: WAITING Blocked count: 1 Waited count: 234 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@51446e33 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1710 (RpcServer.reader=3,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1709 (RpcServer.reader=2,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1708 (RpcServer.reader=1,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1707 (RpcServer.reader=0,bindAddress=localhost,port=50420): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1694 (RpcServer.listener,port=50418): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1705 (RpcServer.responder): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1704 (RpcServer.reader=9,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1703 (RpcServer.reader=8,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1702 (RpcServer.reader=7,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1701 (RpcServer.reader=6,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1700 (RpcServer.reader=5,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1699 (RpcServer.reader=4,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1698 (RpcServer.reader=3,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1697 (RpcServer.reader=2,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1696 (RpcServer.reader=1,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1695 (RpcServer.reader=0,bindAddress=localhost,port=50418): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1681 (FifoRpcScheduler.handler125-thread-1): State: WAITING Blocked count: 1 Waited count: 216 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2818aa9 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1680 (FifoRpcScheduler.handler124-thread-1): State: WAITING Blocked count: 1 Waited count: 191 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2ef83629 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1668 (RpcServer.listener,port=50414): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1679 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1678 (RpcServer.reader=9,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1677 (RpcServer.reader=8,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1676 (RpcServer.reader=7,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1675 (RpcServer.reader=6,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1674 (RpcServer.reader=5,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1673 (RpcServer.reader=4,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1672 (RpcServer.reader=3,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1671 (RpcServer.reader=2,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1670 (RpcServer.reader=1,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1669 (RpcServer.reader=0,bindAddress=localhost,port=50414): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1656 (RpcServer.listener,port=50413): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1667 (RpcServer.responder): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1666 (RpcServer.reader=9,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1665 (RpcServer.reader=8,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1664 (RpcServer.reader=7,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1663 (RpcServer.reader=6,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1662 (RpcServer.reader=5,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1661 (RpcServer.reader=4,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1660 (RpcServer.reader=3,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1659 (RpcServer.reader=2,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1658 (RpcServer.reader=1,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1657 (RpcServer.reader=0,bindAddress=localhost,port=50413): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1644 (RpcServer.listener,port=50412): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1655 (RpcServer.responder): State: RUNNABLE Blocked count: 9 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1654 (RpcServer.reader=9,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1653 (RpcServer.reader=8,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1652 (RpcServer.reader=7,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1651 (RpcServer.reader=6,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1650 (RpcServer.reader=5,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1649 (RpcServer.reader=4,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1648 (RpcServer.reader=3,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1647 (RpcServer.reader=2,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1646 (RpcServer.reader=1,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1645 (RpcServer.reader=0,bindAddress=localhost,port=50412): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1643 (FifoRpcScheduler.handler120-thread-1): State: WAITING Blocked count: 2 Waited count: 220 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74113d57 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1642 (FifoRpcScheduler.handler122-thread-1): State: WAITING Blocked count: 0 Waited count: 176 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5fff560c Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1641 (FifoRpcScheduler.handler121-thread-1): State: WAITING Blocked count: 0 Waited count: 234 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7f7b74a4 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1629 (RpcServer.listener,port=50408): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1640 (RpcServer.responder): State: RUNNABLE Blocked count: 12 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1639 (RpcServer.reader=9,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1638 (RpcServer.reader=8,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1637 (RpcServer.reader=7,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1636 (RpcServer.reader=6,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1635 (RpcServer.reader=5,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1634 (RpcServer.reader=4,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1633 (RpcServer.reader=3,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1632 (RpcServer.reader=2,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1631 (RpcServer.reader=1,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1630 (RpcServer.reader=0,bindAddress=localhost,port=50408): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1617 (RpcServer.listener,port=50407): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1628 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1627 (RpcServer.reader=9,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1626 (RpcServer.reader=8,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1625 (RpcServer.reader=7,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1624 (RpcServer.reader=6,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1623 (RpcServer.reader=5,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1622 (RpcServer.reader=4,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1621 (RpcServer.reader=3,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1620 (RpcServer.reader=2,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1619 (RpcServer.reader=1,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1618 (RpcServer.reader=0,bindAddress=localhost,port=50407): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1616 (FifoRpcScheduler.handler117-thread-1): State: WAITING Blocked count: 0 Waited count: 213 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@36408ffe Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1615 (FifoRpcScheduler.handler118-thread-1): State: WAITING Blocked count: 0 Waited count: 229 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1e5b22e2 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1602 (RpcServer.listener,port=50404): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1614 (RpcServer.responder): State: RUNNABLE Blocked count: 6 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1613 (RpcServer.reader=9,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1612 (RpcServer.reader=8,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1611 (RpcServer.reader=7,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1610 (RpcServer.reader=6,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1609 (RpcServer.reader=5,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1608 (RpcServer.reader=4,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1607 (RpcServer.reader=3,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1606 (RpcServer.reader=2,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1604 (RpcServer.reader=1,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1605 (FifoRpcScheduler.handler119-thread-1): State: WAITING Blocked count: 3 Waited count: 210 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@30e3d58b Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1603 (RpcServer.reader=0,bindAddress=localhost,port=50404): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1590 (RpcServer.listener,port=50402): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1601 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1600 (RpcServer.reader=9,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1599 (RpcServer.reader=8,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1598 (RpcServer.reader=7,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1597 (RpcServer.reader=6,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1596 (RpcServer.reader=5,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1595 (RpcServer.reader=4,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1594 (RpcServer.reader=3,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1593 (RpcServer.reader=2,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1592 (RpcServer.reader=1,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1591 (RpcServer.reader=0,bindAddress=localhost,port=50402): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1578 (RpcServer.listener,port=50401): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1589 (RpcServer.responder): State: RUNNABLE Blocked count: 6 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1588 (RpcServer.reader=9,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1587 (RpcServer.reader=8,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1586 (RpcServer.reader=7,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1585 (RpcServer.reader=6,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1584 (RpcServer.reader=5,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1583 (RpcServer.reader=4,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1582 (RpcServer.reader=3,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1581 (RpcServer.reader=2,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1580 (RpcServer.reader=1,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1579 (RpcServer.reader=0,bindAddress=localhost,port=50401): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1566 (RpcServer.listener,port=50400): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1577 (RpcServer.responder): State: RUNNABLE Blocked count: 7 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1576 (RpcServer.reader=9,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1575 (RpcServer.reader=8,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1574 (RpcServer.reader=7,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1573 (RpcServer.reader=6,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1572 (RpcServer.reader=5,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1571 (RpcServer.reader=4,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1570 (RpcServer.reader=3,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1569 (RpcServer.reader=2,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1568 (RpcServer.reader=1,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1567 (RpcServer.reader=0,bindAddress=localhost,port=50400): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1565 (FifoRpcScheduler.handler116-thread-1): State: WAITING Blocked count: 0 Waited count: 218 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@648b6d8f Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1553 (RpcServer.listener,port=50398): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1564 (RpcServer.responder): State: RUNNABLE Blocked count: 6 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1563 (RpcServer.reader=9,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1562 (RpcServer.reader=8,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1561 (RpcServer.reader=7,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1560 (RpcServer.reader=6,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1559 (RpcServer.reader=5,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1558 (RpcServer.reader=4,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1557 (RpcServer.reader=3,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1556 (RpcServer.reader=2,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1555 (RpcServer.reader=1,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1554 (RpcServer.reader=0,bindAddress=localhost,port=50398): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1539 (FifoRpcScheduler.handler113-thread-1): State: WAITING Blocked count: 0 Waited count: 258 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3ca2c27 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1538 (FifoRpcScheduler.handler114-thread-1): State: WAITING Blocked count: 1 Waited count: 224 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@22b9058a Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1526 (RpcServer.listener,port=50393): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1537 (RpcServer.responder): State: RUNNABLE Blocked count: 7 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1536 (RpcServer.reader=9,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1535 (RpcServer.reader=8,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1534 (RpcServer.reader=7,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1533 (RpcServer.reader=6,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1532 (RpcServer.reader=5,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1531 (RpcServer.reader=4,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1530 (RpcServer.reader=3,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1529 (RpcServer.reader=2,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1528 (RpcServer.reader=1,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1527 (RpcServer.reader=0,bindAddress=localhost,port=50393): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1525 (FifoRpcScheduler.handler112-thread-1): State: WAITING Blocked count: 0 Waited count: 225 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@71d6b3a6 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1513 (RpcServer.listener,port=50392): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1524 (RpcServer.responder): State: RUNNABLE Blocked count: 22 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1523 (RpcServer.reader=9,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1522 (RpcServer.reader=8,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1521 (RpcServer.reader=7,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1520 (RpcServer.reader=6,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1519 (RpcServer.reader=5,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1518 (RpcServer.reader=4,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1517 (RpcServer.reader=3,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1516 (RpcServer.reader=2,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1515 (RpcServer.reader=1,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1514 (RpcServer.reader=0,bindAddress=localhost,port=50392): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1512 (FifoRpcScheduler.handler111-thread-1): State: WAITING Blocked count: 1 Waited count: 217 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@41cd133a Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1500 (RpcServer.listener,port=50389): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1511 (RpcServer.responder): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1510 (RpcServer.reader=9,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1509 (RpcServer.reader=8,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1508 (RpcServer.reader=7,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1507 (RpcServer.reader=6,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1506 (RpcServer.reader=5,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1505 (RpcServer.reader=4,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1504 (RpcServer.reader=3,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1503 (RpcServer.reader=2,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1502 (RpcServer.reader=1,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1501 (RpcServer.reader=0,bindAddress=localhost,port=50389): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1499 (FifoRpcScheduler.handler110-thread-1): State: WAITING Blocked count: 0 Waited count: 230 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@19cd3867 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1487 (RpcServer.listener,port=50387): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1498 (RpcServer.responder): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1497 (RpcServer.reader=9,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1496 (RpcServer.reader=8,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1495 (RpcServer.reader=7,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1494 (RpcServer.reader=6,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1493 (RpcServer.reader=5,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1492 (RpcServer.reader=4,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1491 (RpcServer.reader=3,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1490 (RpcServer.reader=2,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1489 (RpcServer.reader=1,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1488 (RpcServer.reader=0,bindAddress=localhost,port=50387): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1486 (FifoRpcScheduler.handler109-thread-1): State: WAITING Blocked count: 1 Waited count: 259 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@115e237a Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1474 (RpcServer.listener,port=50385): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1485 (RpcServer.responder): State: RUNNABLE Blocked count: 13 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1484 (RpcServer.reader=9,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1483 (RpcServer.reader=8,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1482 (RpcServer.reader=7,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1481 (RpcServer.reader=6,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1480 (RpcServer.reader=5,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1479 (RpcServer.reader=4,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1478 (RpcServer.reader=3,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1477 (RpcServer.reader=2,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1476 (RpcServer.reader=1,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1475 (RpcServer.reader=0,bindAddress=localhost,port=50385): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1462 (RpcServer.listener,port=50384): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1473 (RpcServer.responder): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1472 (RpcServer.reader=9,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1471 (RpcServer.reader=8,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1470 (RpcServer.reader=7,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1469 (RpcServer.reader=6,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1468 (RpcServer.reader=5,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1467 (RpcServer.reader=4,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1466 (RpcServer.reader=3,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1465 (RpcServer.reader=2,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1464 (RpcServer.reader=1,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1463 (RpcServer.reader=0,bindAddress=localhost,port=50384): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1461 (FifoRpcScheduler.handler108-thread-1): State: WAITING Blocked count: 0 Waited count: 258 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@28f3c842 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1449 (RpcServer.listener,port=50382): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1460 (RpcServer.responder): State: RUNNABLE Blocked count: 11 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1459 (RpcServer.reader=9,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1458 (RpcServer.reader=8,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1457 (RpcServer.reader=7,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1456 (RpcServer.reader=6,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1455 (RpcServer.reader=5,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1454 (RpcServer.reader=4,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1453 (RpcServer.reader=3,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1452 (RpcServer.reader=2,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1451 (RpcServer.reader=1,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1450 (RpcServer.reader=0,bindAddress=localhost,port=50382): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1448 (FifoRpcScheduler.handler107-thread-1): State: WAITING Blocked count: 0 Waited count: 242 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@63d7e44e Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1435 (RpcServer.listener,port=50380): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1447 (RpcServer.responder): State: RUNNABLE Blocked count: 12 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1445 (RpcServer.reader=9,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1446 (FifoRpcScheduler.handler105-thread-1): State: WAITING Blocked count: 1 Waited count: 233 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4889ed6 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1444 (RpcServer.reader=8,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1443 (RpcServer.reader=7,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1442 (RpcServer.reader=6,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1441 (RpcServer.reader=5,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1440 (RpcServer.reader=4,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1439 (RpcServer.reader=3,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1438 (RpcServer.reader=2,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1437 (RpcServer.reader=1,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1436 (RpcServer.reader=0,bindAddress=localhost,port=50380): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1434 (FifoRpcScheduler.handler104-thread-1): State: WAITING Blocked count: 0 Waited count: 252 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7e6e0a0 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1409 (RpcServer.listener,port=50375): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1420 (RpcServer.responder): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1419 (RpcServer.reader=9,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1418 (RpcServer.reader=8,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1417 (RpcServer.reader=7,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1416 (RpcServer.reader=6,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1415 (RpcServer.reader=5,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1414 (RpcServer.reader=4,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1413 (RpcServer.reader=3,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1412 (RpcServer.reader=2,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1411 (RpcServer.reader=1,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1410 (RpcServer.reader=0,bindAddress=localhost,port=50375): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1397 (RpcServer.listener,port=50374): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1408 (RpcServer.responder): State: RUNNABLE Blocked count: 13 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1407 (RpcServer.reader=9,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1406 (RpcServer.reader=8,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1405 (RpcServer.reader=7,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1404 (RpcServer.reader=6,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1403 (RpcServer.reader=5,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1402 (RpcServer.reader=4,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1401 (RpcServer.reader=3,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1400 (RpcServer.reader=2,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1399 (RpcServer.reader=1,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1398 (RpcServer.reader=0,bindAddress=localhost,port=50374): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1396 (FifoRpcScheduler.handler103-thread-1): State: WAITING Blocked count: 2 Waited count: 268 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@f03d75b Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1395 (FifoRpcScheduler.handler101-thread-1): State: WAITING Blocked count: 0 Waited count: 230 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3973f6ec Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1394 (FifoRpcScheduler.handler102-thread-1): State: WAITING Blocked count: 3 Waited count: 264 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@cf3935f Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1382 (RpcServer.listener,port=50370): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1393 (RpcServer.responder): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1392 (RpcServer.reader=9,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1391 (RpcServer.reader=8,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1390 (RpcServer.reader=7,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1389 (RpcServer.reader=6,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1388 (RpcServer.reader=5,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1387 (RpcServer.reader=4,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1386 (RpcServer.reader=3,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1385 (RpcServer.reader=2,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1384 (RpcServer.reader=1,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1383 (RpcServer.reader=0,bindAddress=localhost,port=50370): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1370 (RpcServer.listener,port=50369): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1381 (RpcServer.responder): State: RUNNABLE Blocked count: 6 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1380 (RpcServer.reader=9,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1379 (RpcServer.reader=8,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1378 (RpcServer.reader=7,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1377 (RpcServer.reader=6,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1376 (RpcServer.reader=5,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1375 (RpcServer.reader=4,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1374 (RpcServer.reader=3,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1373 (RpcServer.reader=2,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1372 (RpcServer.reader=1,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1371 (RpcServer.reader=0,bindAddress=localhost,port=50369): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1358 (RpcServer.listener,port=50368): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1369 (RpcServer.responder): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1368 (RpcServer.reader=9,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1367 (RpcServer.reader=8,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1366 (RpcServer.reader=7,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1365 (RpcServer.reader=6,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1364 (RpcServer.reader=5,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1363 (RpcServer.reader=4,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1362 (RpcServer.reader=3,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1361 (RpcServer.reader=2,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1360 (RpcServer.reader=1,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1359 (RpcServer.reader=0,bindAddress=localhost,port=50368): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1357 (FifoRpcScheduler.handler100-thread-1): State: WAITING Blocked count: 2 Waited count: 236 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1954bbc1 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1344 (RpcServer.listener,port=50364): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1356 (RpcServer.responder): State: RUNNABLE Blocked count: 13 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1355 (RpcServer.reader=9,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1354 (RpcServer.reader=8,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1353 (RpcServer.reader=7,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1352 (RpcServer.reader=6,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1351 (RpcServer.reader=5,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1350 (RpcServer.reader=4,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1349 (RpcServer.reader=3,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1348 (RpcServer.reader=2,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1346 (RpcServer.reader=1,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1347 (FifoRpcScheduler.handler99-thread-1): State: WAITING Blocked count: 0 Waited count: 263 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1155fcb5 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1345 (RpcServer.reader=0,bindAddress=localhost,port=50364): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1331 (RpcServer.listener,port=50361): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1342 (RpcServer.responder): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1341 (RpcServer.reader=9,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1340 (RpcServer.reader=8,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1339 (RpcServer.reader=7,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1338 (RpcServer.reader=6,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1337 (RpcServer.reader=5,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1336 (RpcServer.reader=4,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1335 (RpcServer.reader=3,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1334 (RpcServer.reader=2,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1333 (RpcServer.reader=1,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1332 (RpcServer.reader=0,bindAddress=localhost,port=50361): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1305 (FifoRpcScheduler.handler96-thread-1): State: WAITING Blocked count: 1 Waited count: 261 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@24c360d5 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1304 (FifoRpcScheduler.handler95-thread-1): State: WAITING Blocked count: 0 Waited count: 234 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@26c25e33 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1292 (RpcServer.listener,port=50355): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1303 (RpcServer.responder): State: RUNNABLE Blocked count: 12 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1302 (RpcServer.reader=9,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1301 (RpcServer.reader=8,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1300 (RpcServer.reader=7,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1299 (RpcServer.reader=6,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1298 (RpcServer.reader=5,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1297 (RpcServer.reader=4,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1296 (RpcServer.reader=3,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1295 (RpcServer.reader=2,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1294 (RpcServer.reader=1,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1293 (RpcServer.reader=0,bindAddress=localhost,port=50355): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1280 (RpcServer.listener,port=50352): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1291 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1290 (RpcServer.reader=9,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1289 (RpcServer.reader=8,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1288 (RpcServer.reader=7,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1287 (RpcServer.reader=6,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1286 (RpcServer.reader=5,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1285 (RpcServer.reader=4,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1284 (RpcServer.reader=3,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1283 (RpcServer.reader=2,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1282 (RpcServer.reader=1,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1281 (RpcServer.reader=0,bindAddress=localhost,port=50352): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1278 (FifoRpcScheduler.handler94-thread-1): State: WAITING Blocked count: 0 Waited count: 266 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@598ac010 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1277 (FifoRpcScheduler.handler93-thread-1): State: WAITING Blocked count: 0 Waited count: 262 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5449a446 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1276 (FifoRpcScheduler.handler92-thread-1): State: WAITING Blocked count: 2 Waited count: 266 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4625c85d Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1264 (RpcServer.listener,port=50347): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1275 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1274 (RpcServer.reader=9,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1273 (RpcServer.reader=8,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1272 (RpcServer.reader=7,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1271 (RpcServer.reader=6,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1270 (RpcServer.reader=5,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1269 (RpcServer.reader=4,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1268 (RpcServer.reader=3,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1267 (RpcServer.reader=2,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1266 (RpcServer.reader=1,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1265 (RpcServer.reader=0,bindAddress=localhost,port=50347): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1252 (RpcServer.listener,port=50346): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1263 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1262 (RpcServer.reader=9,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1261 (RpcServer.reader=8,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1260 (RpcServer.reader=7,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1259 (RpcServer.reader=6,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1258 (RpcServer.reader=5,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1257 (RpcServer.reader=4,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1256 (RpcServer.reader=3,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1255 (RpcServer.reader=2,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1254 (RpcServer.reader=1,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1253 (RpcServer.reader=0,bindAddress=localhost,port=50346): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1240 (RpcServer.listener,port=50345): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1251 (RpcServer.responder): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1250 (RpcServer.reader=9,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1249 (RpcServer.reader=8,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1248 (RpcServer.reader=7,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1247 (RpcServer.reader=6,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1246 (RpcServer.reader=5,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1245 (RpcServer.reader=4,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1244 (RpcServer.reader=3,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1243 (RpcServer.reader=2,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1242 (RpcServer.reader=1,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1241 (RpcServer.reader=0,bindAddress=localhost,port=50345): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1227 (FifoRpcScheduler.handler90-thread-1): State: WAITING Blocked count: 1 Waited count: 275 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@769d0166 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1215 (RpcServer.listener,port=50340): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1226 (RpcServer.responder): State: RUNNABLE Blocked count: 7 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1225 (RpcServer.reader=9,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1224 (RpcServer.reader=8,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1223 (RpcServer.reader=7,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1222 (RpcServer.reader=6,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1221 (RpcServer.reader=5,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1220 (RpcServer.reader=4,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1219 (RpcServer.reader=3,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1218 (RpcServer.reader=2,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1217 (RpcServer.reader=1,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1216 (RpcServer.reader=0,bindAddress=localhost,port=50340): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1214 (FifoRpcScheduler.handler89-thread-1): State: WAITING Blocked count: 0 Waited count: 281 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@d656e4e Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1213 (FifoRpcScheduler.handler88-thread-1): State: WAITING Blocked count: 0 Waited count: 246 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2e719cf7 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1201 (RpcServer.listener,port=50337): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1212 (RpcServer.responder): State: RUNNABLE Blocked count: 15 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1211 (RpcServer.reader=9,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1210 (RpcServer.reader=8,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1209 (RpcServer.reader=7,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1208 (RpcServer.reader=6,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1207 (RpcServer.reader=5,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1206 (RpcServer.reader=4,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1205 (RpcServer.reader=3,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1204 (RpcServer.reader=2,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1203 (RpcServer.reader=1,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1202 (RpcServer.reader=0,bindAddress=localhost,port=50337): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1189 (RpcServer.listener,port=50335): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1200 (RpcServer.responder): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1199 (RpcServer.reader=9,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1198 (RpcServer.reader=8,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1197 (RpcServer.reader=7,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1196 (RpcServer.reader=6,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1195 (RpcServer.reader=5,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1194 (RpcServer.reader=4,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1193 (RpcServer.reader=3,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1192 (RpcServer.reader=2,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1191 (RpcServer.reader=1,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1190 (RpcServer.reader=0,bindAddress=localhost,port=50335): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1162 (FifoRpcScheduler.handler85-thread-1): State: WAITING Blocked count: 1 Waited count: 245 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@76519645 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1150 (RpcServer.listener,port=50325): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1161 (RpcServer.responder): State: RUNNABLE Blocked count: 5 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1160 (RpcServer.reader=9,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1159 (RpcServer.reader=8,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1158 (RpcServer.reader=7,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1157 (RpcServer.reader=6,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1156 (RpcServer.reader=5,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1155 (RpcServer.reader=4,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1154 (RpcServer.reader=3,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1153 (RpcServer.reader=2,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1152 (RpcServer.reader=1,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1151 (RpcServer.reader=0,bindAddress=localhost,port=50325): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1149 (FifoRpcScheduler.handler84-thread-1): State: WAITING Blocked count: 2 Waited count: 281 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@12656603 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1137 (RpcServer.listener,port=50323): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1148 (RpcServer.responder): State: RUNNABLE Blocked count: 9 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1147 (RpcServer.reader=9,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1146 (RpcServer.reader=8,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1145 (RpcServer.reader=7,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1144 (RpcServer.reader=6,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1143 (RpcServer.reader=5,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1142 (RpcServer.reader=4,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1141 (RpcServer.reader=3,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1140 (RpcServer.reader=2,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1139 (RpcServer.reader=1,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1138 (RpcServer.reader=0,bindAddress=localhost,port=50323): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1072 (FifoRpcScheduler.handler77-thread-1): State: WAITING Blocked count: 0 Waited count: 281 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@12115ae0 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1071 (FifoRpcScheduler.handler78-thread-1): State: WAITING Blocked count: 2 Waited count: 303 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6a8db591 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1059 (RpcServer.listener,port=50303): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1070 (RpcServer.responder): State: RUNNABLE Blocked count: 20 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1069 (RpcServer.reader=9,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1068 (RpcServer.reader=8,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1067 (RpcServer.reader=7,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1066 (RpcServer.reader=6,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1065 (RpcServer.reader=5,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1064 (RpcServer.reader=4,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1063 (RpcServer.reader=3,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1062 (RpcServer.reader=2,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1061 (RpcServer.reader=1,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1060 (RpcServer.reader=0,bindAddress=localhost,port=50303): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1047 (RpcServer.listener,port=50302): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1058 (RpcServer.responder): State: RUNNABLE Blocked count: 8 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1057 (RpcServer.reader=9,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1056 (RpcServer.reader=8,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1055 (RpcServer.reader=7,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1054 (RpcServer.reader=6,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1053 (RpcServer.reader=5,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1052 (RpcServer.reader=4,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1051 (RpcServer.reader=3,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1050 (RpcServer.reader=2,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1049 (RpcServer.reader=1,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1048 (RpcServer.reader=0,bindAddress=localhost,port=50302): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1046 (FifoRpcScheduler.handler76-thread-1): State: WAITING Blocked count: 0 Waited count: 297 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3c4b52b7 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1034 (RpcServer.listener,port=50300): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1045 (RpcServer.responder): State: RUNNABLE Blocked count: 15 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1044 (RpcServer.reader=9,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1043 (RpcServer.reader=8,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1042 (RpcServer.reader=7,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1041 (RpcServer.reader=6,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1040 (RpcServer.reader=5,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1039 (RpcServer.reader=4,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1038 (RpcServer.reader=3,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1037 (RpcServer.reader=2,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1036 (RpcServer.reader=1,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1035 (RpcServer.reader=0,bindAddress=localhost,port=50300): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1032 (FifoRpcScheduler.handler74-thread-1): State: WAITING Blocked count: 2 Waited count: 259 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@33a91cbe Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1008 (RpcServer.listener,port=50294): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 1019 (RpcServer.responder): State: RUNNABLE Blocked count: 13 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 1018 (RpcServer.reader=9,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1017 (RpcServer.reader=8,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1016 (RpcServer.reader=7,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1015 (RpcServer.reader=6,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1014 (RpcServer.reader=5,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1013 (RpcServer.reader=4,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1012 (RpcServer.reader=3,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1011 (RpcServer.reader=2,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1010 (RpcServer.reader=1,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 1009 (RpcServer.reader=0,bindAddress=localhost,port=50294): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 993 (FifoRpcScheduler.handler70-thread-1): State: WAITING Blocked count: 0 Waited count: 294 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7e87cb0f Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 956 (RpcServer.listener,port=50282): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 967 (RpcServer.responder): State: RUNNABLE Blocked count: 16 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 966 (RpcServer.reader=9,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 965 (RpcServer.reader=8,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 964 (RpcServer.reader=7,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 963 (RpcServer.reader=6,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 962 (RpcServer.reader=5,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 961 (RpcServer.reader=4,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 960 (RpcServer.reader=3,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 959 (RpcServer.reader=2,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 958 (RpcServer.reader=1,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 957 (RpcServer.reader=0,bindAddress=localhost,port=50282): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 955 (FifoRpcScheduler.handler69-thread-1): State: WAITING Blocked count: 0 Waited count: 275 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5a7e6f61 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 954 (FifoRpcScheduler.handler66-thread-1): State: WAITING Blocked count: 2 Waited count: 269 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@d40174b Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 942 (RpcServer.listener,port=50279): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 953 (RpcServer.responder): State: RUNNABLE Blocked count: 12 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 952 (RpcServer.reader=9,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 951 (RpcServer.reader=8,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 950 (RpcServer.reader=7,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 949 (RpcServer.reader=6,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 948 (RpcServer.reader=5,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 947 (RpcServer.reader=4,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 946 (RpcServer.reader=3,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 945 (RpcServer.reader=2,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 944 (RpcServer.reader=1,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 943 (RpcServer.reader=0,bindAddress=localhost,port=50279): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 903 (RpcServer.listener,port=50273): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 914 (RpcServer.responder): State: RUNNABLE Blocked count: 10 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 913 (RpcServer.reader=9,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 912 (RpcServer.reader=8,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 911 (RpcServer.reader=7,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 910 (RpcServer.reader=6,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 909 (RpcServer.reader=5,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 908 (RpcServer.reader=4,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 907 (RpcServer.reader=3,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 906 (RpcServer.reader=2,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 905 (RpcServer.reader=1,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 904 (RpcServer.reader=0,bindAddress=localhost,port=50273): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 863 (FifoRpcScheduler.handler61-thread-1): State: WAITING Blocked count: 3 Waited count: 306 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7096036a Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 839 (RpcServer.listener,port=50264): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 850 (RpcServer.responder): State: RUNNABLE Blocked count: 7 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 849 (RpcServer.reader=9,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 848 (RpcServer.reader=8,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 847 (RpcServer.reader=7,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 846 (RpcServer.reader=6,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 845 (RpcServer.reader=5,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 844 (RpcServer.reader=4,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 843 (RpcServer.reader=3,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 842 (RpcServer.reader=2,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 841 (RpcServer.reader=1,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 840 (RpcServer.reader=0,bindAddress=localhost,port=50264): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 824 (FifoRpcScheduler.handler57-thread-1): State: WAITING Blocked count: 1 Waited count: 308 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@570fc04b Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 811 (FifoRpcScheduler.handler58-thread-1): State: WAITING Blocked count: 1 Waited count: 310 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@18ab311 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 799 (RpcServer.listener,port=50257): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 810 (RpcServer.responder): State: RUNNABLE Blocked count: 19 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 809 (RpcServer.reader=9,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 808 (RpcServer.reader=8,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 807 (RpcServer.reader=7,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 806 (RpcServer.reader=6,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 805 (RpcServer.reader=5,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 804 (RpcServer.reader=4,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 803 (RpcServer.reader=3,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 802 (RpcServer.reader=2,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 801 (RpcServer.reader=1,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 800 (RpcServer.reader=0,bindAddress=localhost,port=50257): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 787 (RpcServer.listener,port=50256): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 798 (RpcServer.responder): State: RUNNABLE Blocked count: 15 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 797 (RpcServer.reader=9,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 796 (RpcServer.reader=8,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 795 (RpcServer.reader=7,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 794 (RpcServer.reader=6,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 793 (RpcServer.reader=5,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 792 (RpcServer.reader=4,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 791 (RpcServer.reader=3,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 790 (RpcServer.reader=2,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 789 (RpcServer.reader=1,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 788 (RpcServer.reader=0,bindAddress=localhost,port=50256): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 786 (FifoRpcScheduler.handler56-thread-1): State: WAITING Blocked count: 1 Waited count: 297 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@668ade35 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 774 (RpcServer.listener,port=50254): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 785 (RpcServer.responder): State: RUNNABLE Blocked count: 16 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 784 (RpcServer.reader=9,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 783 (RpcServer.reader=8,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 782 (RpcServer.reader=7,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 781 (RpcServer.reader=6,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 780 (RpcServer.reader=5,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 779 (RpcServer.reader=4,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 778 (RpcServer.reader=3,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 777 (RpcServer.reader=2,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 776 (RpcServer.reader=1,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 775 (RpcServer.reader=0,bindAddress=localhost,port=50254): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 773 (FifoRpcScheduler.handler55-thread-1): State: WAITING Blocked count: 1 Waited count: 296 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4c031ab3 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 761 (RpcServer.listener,port=50252): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 772 (RpcServer.responder): State: RUNNABLE Blocked count: 15 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 771 (RpcServer.reader=9,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 770 (RpcServer.reader=8,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 769 (RpcServer.reader=7,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 768 (RpcServer.reader=6,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 767 (RpcServer.reader=5,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 766 (RpcServer.reader=4,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 765 (RpcServer.reader=3,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 764 (RpcServer.reader=2,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 763 (RpcServer.reader=1,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 762 (RpcServer.reader=0,bindAddress=localhost,port=50252): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 732 (FifoRpcScheduler.handler51-thread-1): State: WAITING Blocked count: 1 Waited count: 282 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ca9fa7 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 708 (RpcServer.listener,port=50243): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 719 (RpcServer.responder): State: RUNNABLE Blocked count: 9 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 718 (RpcServer.reader=9,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 717 (RpcServer.reader=8,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 716 (RpcServer.reader=7,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 715 (RpcServer.reader=6,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 714 (RpcServer.reader=5,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 713 (RpcServer.reader=4,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 712 (RpcServer.reader=3,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 711 (RpcServer.reader=2,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 710 (RpcServer.reader=1,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 709 (RpcServer.reader=0,bindAddress=localhost,port=50243): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 694 (FifoRpcScheduler.handler49-thread-1): State: WAITING Blocked count: 0 Waited count: 336 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4081884a Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 682 (RpcServer.listener,port=50239): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 693 (RpcServer.responder): State: RUNNABLE Blocked count: 21 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 692 (RpcServer.reader=9,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 691 (RpcServer.reader=8,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 690 (RpcServer.reader=7,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 689 (RpcServer.reader=6,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 688 (RpcServer.reader=5,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 687 (RpcServer.reader=4,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 686 (RpcServer.reader=3,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 685 (RpcServer.reader=2,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 684 (RpcServer.reader=1,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 683 (RpcServer.reader=0,bindAddress=localhost,port=50239): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 643 (FifoRpcScheduler.handler45-thread-1): State: WAITING Blocked count: 1 Waited count: 353 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6cb3e9f4 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 631 (RpcServer.listener,port=50232): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 642 (RpcServer.responder): State: RUNNABLE Blocked count: 13 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 641 (RpcServer.reader=9,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 640 (RpcServer.reader=8,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 639 (RpcServer.reader=7,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 638 (RpcServer.reader=6,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 637 (RpcServer.reader=5,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 636 (RpcServer.reader=4,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 635 (RpcServer.reader=3,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 634 (RpcServer.reader=2,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 633 (RpcServer.reader=1,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 632 (RpcServer.reader=0,bindAddress=localhost,port=50232): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 527 (FifoRpcScheduler.handler36-thread-1): State: WAITING Blocked count: 0 Waited count: 326 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@66aa5740 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 515 (RpcServer.listener,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 526 (RpcServer.responder): State: RUNNABLE Blocked count: 12 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 525 (RpcServer.reader=9,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 524 (RpcServer.reader=8,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 523 (RpcServer.reader=7,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 522 (RpcServer.reader=6,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 521 (RpcServer.reader=5,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 520 (RpcServer.reader=4,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 519 (RpcServer.reader=3,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 518 (RpcServer.reader=2,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 517 (RpcServer.reader=1,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 516 (RpcServer.reader=0,bindAddress=localhost,port=50214): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 475 (FifoRpcScheduler.handler32-thread-1): State: WAITING Blocked count: 2 Waited count: 365 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3e2790e0 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 463 (RpcServer.listener,port=50206): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 474 (RpcServer.responder): State: RUNNABLE Blocked count: 18 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 473 (RpcServer.reader=9,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 472 (RpcServer.reader=8,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 471 (RpcServer.reader=7,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 470 (RpcServer.reader=6,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 469 (RpcServer.reader=5,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 468 (RpcServer.reader=4,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 467 (RpcServer.reader=3,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 466 (RpcServer.reader=2,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 465 (RpcServer.reader=1,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 464 (RpcServer.reader=0,bindAddress=localhost,port=50206): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 424 (FifoRpcScheduler.handler27-thread-1): State: WAITING Blocked count: 0 Waited count: 356 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@20726dad Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 423 (FifoRpcScheduler.handler28-thread-1): State: WAITING Blocked count: 3 Waited count: 350 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@41de9550 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 411 (RpcServer.listener,port=50198): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 422 (RpcServer.responder): State: RUNNABLE Blocked count: 17 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 421 (RpcServer.reader=9,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 420 (RpcServer.reader=8,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 419 (RpcServer.reader=7,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 418 (RpcServer.reader=6,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 417 (RpcServer.reader=5,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 416 (RpcServer.reader=4,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 415 (RpcServer.reader=3,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 414 (RpcServer.reader=2,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 413 (RpcServer.reader=1,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 412 (RpcServer.reader=0,bindAddress=localhost,port=50198): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 399 (RpcServer.listener,port=50197): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 410 (RpcServer.responder): State: RUNNABLE Blocked count: 16 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 409 (RpcServer.reader=9,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 408 (RpcServer.reader=8,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 407 (RpcServer.reader=7,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 406 (RpcServer.reader=6,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 405 (RpcServer.reader=5,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 404 (RpcServer.reader=4,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 403 (RpcServer.reader=3,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 402 (RpcServer.reader=2,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 401 (RpcServer.reader=1,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 400 (RpcServer.reader=0,bindAddress=localhost,port=50197): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 307 (FifoRpcScheduler.handler18-thread-1): State: WAITING Blocked count: 5 Waited count: 350 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@44e79b9a Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 294 (FifoRpcScheduler.handler17-thread-1): State: WAITING Blocked count: 1 Waited count: 371 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1316a660 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 282 (RpcServer.listener,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 293 (RpcServer.responder): State: RUNNABLE Blocked count: 22 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 292 (RpcServer.reader=9,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 291 (RpcServer.reader=8,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 290 (RpcServer.reader=7,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 289 (RpcServer.reader=6,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 288 (RpcServer.reader=5,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 287 (RpcServer.reader=4,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 286 (RpcServer.reader=3,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 285 (RpcServer.reader=2,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 284 (RpcServer.reader=1,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 283 (RpcServer.reader=0,bindAddress=localhost,port=50180): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 149 (AsyncRpcChannel-pool2-t16): State: RUNNABLE Blocked count: 151 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 269 (RpcServer.listener,port=50177): State: RUNNABLE Blocked count: 2 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 280 (RpcServer.responder): State: RUNNABLE Blocked count: 18 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 279 (RpcServer.reader=9,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 278 (RpcServer.reader=8,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 277 (RpcServer.reader=7,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 276 (RpcServer.reader=6,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 275 (RpcServer.reader=5,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 274 (RpcServer.reader=4,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 273 (RpcServer.reader=3,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 272 (RpcServer.reader=2,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 271 (RpcServer.reader=1,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 1 Waited count: 1 Stack: sun.nio.ch.FileDispatcherImpl.read0(Native Method) sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) sun.nio.ch.IOUtil.read(IOUtil.java:197) sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) org.apache.hadoop.hbase.ipc.RpcServer.channelRead(RpcServer.java:2363) org.apache.hadoop.hbase.ipc.RpcServer$Connection.readAndProcess(RpcServer.java:1607) org.apache.hadoop.hbase.ipc.RpcServer$Listener.doRead(RpcServer.java:872) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:657) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 270 (RpcServer.reader=0,bindAddress=localhost,port=50177): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 256 (FifoRpcScheduler.handler15-thread-1): State: WAITING Blocked count: 1 Waited count: 386 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5acfda6b Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 148 (AsyncRpcChannel-pool2-t15): State: RUNNABLE Blocked count: 119 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 244 (RpcServer.listener,port=50174): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 255 (RpcServer.responder): State: RUNNABLE Blocked count: 27 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 254 (RpcServer.reader=9,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 253 (RpcServer.reader=8,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 252 (RpcServer.reader=7,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 251 (RpcServer.reader=6,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 250 (RpcServer.reader=5,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 249 (RpcServer.reader=4,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 248 (RpcServer.reader=3,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 247 (RpcServer.reader=2,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 246 (RpcServer.reader=1,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 245 (RpcServer.reader=0,bindAddress=localhost,port=50174): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 243 (FifoRpcScheduler.handler14-thread-1): State: WAITING Blocked count: 0 Waited count: 415 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@568ee2b9 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 147 (AsyncRpcChannel-pool2-t14): State: RUNNABLE Blocked count: 99 Waited count: 0 Stack: com.google.protobuf.CodedInputStream.readRawBytes(CodedInputStream.java:874) com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:329) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto.(TestProtos.java:1275) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto.(TestProtos.java:1225) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$1.parsePartialFrom(TestProtos.java:1308) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$1.parsePartialFrom(TestProtos.java:1303) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$Builder.mergeFrom(TestProtos.java:1623) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$Builder.mergeFrom(TestProtos.java:1515) com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337) com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267) org.apache.hadoop.hbase.protobuf.ProtobufUtil.mergeDelimitedFrom(ProtobufUtil.java:3025) org.apache.hadoop.hbase.ipc.AsyncServerResponseHandler.channelRead(AsyncServerResponseHandler.java:90) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) Thread 231 (RpcServer.listener,port=50172): State: RUNNABLE Blocked count: 1 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 242 (RpcServer.responder): State: RUNNABLE Blocked count: 24 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 241 (RpcServer.reader=9,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 240 (RpcServer.reader=8,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 239 (RpcServer.reader=7,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 238 (RpcServer.reader=6,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 237 (RpcServer.reader=5,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 236 (RpcServer.reader=4,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 235 (RpcServer.reader=3,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 234 (RpcServer.reader=2,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 233 (RpcServer.reader=1,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 2 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 232 (RpcServer.reader=0,bindAddress=localhost,port=50172): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 146 (AsyncRpcChannel-pool2-t13): State: RUNNABLE Blocked count: 129 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 145 (AsyncRpcChannel-pool2-t12): State: RUNNABLE Blocked count: 121 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 222 (FifoRpcScheduler.handler3-thread-1): State: WAITING Blocked count: 3 Waited count: 403 Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5185b802 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:374) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 144 (AsyncRpcChannel-pool2-t11): State: RUNNABLE Blocked count: 90 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 143 (AsyncRpcChannel-pool2-t10): State: RUNNABLE Blocked count: 172 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 142 (AsyncRpcChannel-pool2-t9): State: RUNNABLE Blocked count: 181 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 141 (AsyncRpcChannel-pool2-t8): State: RUNNABLE Blocked count: 144 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 140 (AsyncRpcChannel-pool2-t7): State: RUNNABLE Blocked count: 134 Waited count: 0 Stack: io.netty.buffer.UnpooledUnsafeDirectByteBuf.capacity(UnpooledUnsafeDirectByteBuf.java:160) io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:251) io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:849) io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:841) io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:831) io.netty.handler.codec.ByteToMessageDecoder$1.cumulate(ByteToMessageDecoder.java:92) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:228) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 139 (AsyncRpcChannel-pool2-t6): State: RUNNABLE Blocked count: 167 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 138 (AsyncRpcChannel-pool2-t5): State: RUNNABLE Blocked count: 115 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 137 (AsyncRpcChannel-pool2-t4): State: RUNNABLE Blocked count: 167 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 136 (AsyncRpcChannel-pool2-t3): State: RUNNABLE Blocked count: 185 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 135 (AsyncRpcChannel-pool2-t2): State: RUNNABLE Blocked count: 123 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:622) io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310) io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:110) java.lang.Thread.run(Thread.java:745) Thread 134 (AsyncRpcChannel-pool2-t1): State: RUNNABLE Blocked count: 137 Waited count: 0 Stack: com.google.protobuf.CodedInputStream.readRawBytes(CodedInputStream.java:874) com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:329) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto.(TestProtos.java:1275) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto.(TestProtos.java:1225) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$1.parsePartialFrom(TestProtos.java:1308) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$1.parsePartialFrom(TestProtos.java:1303) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$Builder.mergeFrom(TestProtos.java:1623) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto$Builder.mergeFrom(TestProtos.java:1515) com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337) com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267) org.apache.hadoop.hbase.protobuf.ProtobufUtil.mergeDelimitedFrom(ProtobufUtil.java:3025) org.apache.hadoop.hbase.ipc.AsyncServerResponseHandler.channelRead(AsyncServerResponseHandler.java:90) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244) io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308) io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294) io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846) io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511) Thread 179 (Thread-42): State: WAITING Blocked count: 34 Waited count: 30 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@30e04dea Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 178 (Thread-41): State: WAITING Blocked count: 1874 Waited count: 1774 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@1cda348b Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 177 (Thread-40): State: WAITING Blocked count: 36 Waited count: 31 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@39508585 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 176 (Thread-39): State: WAITING Blocked count: 50 Waited count: 50 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@2faae4d0 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 175 (Thread-38): State: WAITING Blocked count: 371 Waited count: 369 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@63ddac13 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 174 (Thread-37): State: WAITING Blocked count: 1950 Waited count: 1845 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@51b025fd Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 173 (Thread-36): State: WAITING Blocked count: 5 Waited count: 2 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@6babd209 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 172 (Thread-35): State: WAITING Blocked count: 29 Waited count: 22 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@7c067391 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 171 (Thread-34): State: WAITING Blocked count: 259 Waited count: 237 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@73c807d8 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 170 (Thread-33): State: WAITING Blocked count: 106 Waited count: 103 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@6ee11b10 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 169 (Thread-32): State: WAITING Blocked count: 357 Waited count: 361 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@1001d0da Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 168 (Thread-31): State: WAITING Blocked count: 1922 Waited count: 1830 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@335e05bd Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 167 (Thread-30): State: WAITING Blocked count: 25 Waited count: 20 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@4b8ff27d Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 166 (Thread-29): State: RUNNABLE Blocked count: 1913 Waited count: 1817 Stack: java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149) java.lang.StringCoding.decode(StringCoding.java:193) java.lang.String.(String.java:416) com.google.protobuf.LiteralByteString.toString(LiteralByteString.java:148) com.google.protobuf.ByteString.toStringUtf8(ByteString.java:572) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoResponseProto.getMessage(TestProtos.java:1337) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:316) Thread 165 (Thread-28): State: WAITING Blocked count: 65 Waited count: 64 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@33ca8a02 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 164 (Thread-27): State: WAITING Blocked count: 393 Waited count: 397 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@2276fe72 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 163 (Thread-26): State: WAITING Blocked count: 1949 Waited count: 1840 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@1e7f4e07 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 162 (Thread-25): State: WAITING Blocked count: 1867 Waited count: 1749 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@78fdc35b Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 161 (Thread-24): State: WAITING Blocked count: 33 Waited count: 26 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@e49bfad Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 160 (Thread-23): State: WAITING Blocked count: 1838 Waited count: 1749 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@60f147cd Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 159 (Thread-22): State: WAITING Blocked count: 39 Waited count: 33 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@4f8772fe Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 158 (Thread-21): State: WAITING Blocked count: 1876 Waited count: 1801 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@72c5b7be Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 157 (Thread-20): State: RUNNABLE Blocked count: 1886 Waited count: 1793 Stack: java.lang.StringCoding$StringEncoder.encode(StringCoding.java:304) java.lang.StringCoding.encode(StringCoding.java:344) java.lang.String.getBytes(String.java:916) com.google.protobuf.ByteString.copyFromUtf8(ByteString.java:255) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoRequestProto.getMessageBytes(TestProtos.java:834) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoRequestProto.getSerializedSize(TestProtos.java:876) org.apache.hadoop.hbase.ipc.IPCUtil.getTotalSizeWhenWrittenDelimited(IPCUtil.java:306) org.apache.hadoop.hbase.ipc.AsyncRpcChannel.writeRequest(AsyncRpcChannel.java:430) org.apache.hadoop.hbase.ipc.AsyncRpcChannel.callMethod(AsyncRpcChannel.java:351) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:245) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 156 (Thread-19): State: WAITING Blocked count: 1916 Waited count: 1813 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@6d763be1 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 155 (Thread-18): State: WAITING Blocked count: 6 Waited count: 2 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@25b5461 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 154 (Thread-17): State: WAITING Blocked count: 87 Waited count: 83 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@4c9a500d Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 153 (Thread-16): State: RUNNABLE Blocked count: 1841 Waited count: 1760 Stack: org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:300) Thread 152 (Thread-15): State: WAITING Blocked count: 54 Waited count: 49 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@e457208 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 151 (Thread-14): State: RUNNABLE Blocked count: 2024 Waited count: 1897 Stack: java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300) java.lang.StringCoding.encode(StringCoding.java:344) java.lang.String.getBytes(String.java:916) com.google.protobuf.ByteString.copyFromUtf8(ByteString.java:255) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoRequestProto.getMessageBytes(TestProtos.java:834) org.apache.hadoop.hbase.ipc.protobuf.generated.TestProtos$EchoRequestProto.getSerializedSize(TestProtos.java:876) org.apache.hadoop.hbase.ipc.IPCUtil.getTotalSizeWhenWrittenDelimited(IPCUtil.java:306) org.apache.hadoop.hbase.ipc.AsyncRpcChannel.writeRequest(AsyncRpcChannel.java:430) org.apache.hadoop.hbase.ipc.AsyncRpcChannel.callMethod(AsyncRpcChannel.java:351) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:245) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 150 (Thread-13): State: WAITING Blocked count: 8 Waited count: 3 Waiting on org.apache.hadoop.hbase.ipc.AsyncCall@58d88305 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:254) io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:32) io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:31) org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:248) org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient.run(IntegrationTestRpcClient.java:307) Thread 37 (RpcServer.listener,port=50148): State: RUNNABLE Blocked count: 4 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:749) Thread 48 (RpcServer.responder): State: RUNNABLE Blocked count: 31 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:988) org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:935) Thread 47 (RpcServer.reader=9,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 46 (RpcServer.reader=8,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 45 (RpcServer.reader=7,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 44 (RpcServer.reader=6,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 43 (RpcServer.reader=5,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 42 (RpcServer.reader=4,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 41 (RpcServer.reader=3,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 40 (RpcServer.reader=2,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 39 (RpcServer.reader=1,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 10 Waited count: 1 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 38 (RpcServer.reader=0,bindAddress=localhost,port=50148): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method) sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:200) sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:646) org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:633) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:745) Thread 23 (Timer for 'HBase' metrics system): State: TIMED_WAITING Blocked count: 1 Waited count: 11 Stack: java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Thread 11 (Thread-1): State: RUNNABLE Blocked count: 0 Waited count: 1 Stack: sun.management.ThreadImpl.getThreadInfo1(Native Method) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:174) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:139) org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:168) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) org.apache.hadoop.hbase.util.Threads$PrintThreadInfoLazyHolder$1.printThreadInfo(Threads.java:294) org.apache.hadoop.hbase.util.Threads.printThreadInfo(Threads.java:339) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$TimeoutThread.run(IntegrationTestRpcClient.java:389) Thread 10 (Time-limited test): State: WAITING Blocked count: 9 Waited count: 3 Waiting on org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$SimpleClient@458a2f35 Stack: java.lang.Object.wait(Native Method) java.lang.Thread.join(Thread.java:1281) java.lang.Thread.join(Thread.java:1355) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient.testRpcWithChaosMonkey(IntegrationTestRpcClient.java:442) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$4.call(IntegrationTestRpcClient.java:365) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$4.call(IntegrationTestRpcClient.java:361) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient$TimeoutThread.runWithTimeout(IntegrationTestRpcClient.java:401) org.apache.hadoop.hbase.ipc.IntegrationTestRpcClient.testRpcWithChaosMonkeyWithAsyncClient(IntegrationTestRpcClient.java:361) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:606) org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) java.util.concurrent.FutureTask.run(FutureTask.java:262) java.lang.Thread.run(Thread.java:745) Thread 5 (Signal Dispatcher): State: RUNNABLE Blocked count: 0 Waited count: 0 Stack: Thread 3 (Finalizer): State: WAITING Blocked count: 636 Waited count: 58 Waiting on java.lang.ref.ReferenceQueue$Lock@4cac0dda Stack: java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) Thread 2 (Reference Handler): State: WAITING Blocked count: 239 Waited count: 212 Waiting on java.lang.ref.Reference$Lock@33c18a89 Stack: java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:503) java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) Thread 1 (main): State: TIMED_WAITING Blocked count: 0 Waited count: 2 Stack: sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) java.util.concurrent.FutureTask.awaitDone(FutureTask.java:422) java.util.concurrent.FutureTask.get(FutureTask.java:199) org.junit.internal.runners.statements.FailOnTimeout.getResult(FailOnTimeout.java:141) org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.java:127) org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) org.junit.runners.ParentRunner.run(ParentRunner.java:363) org.junit.runners.Suite.runChild(Suite.java:128) org.junit.runners.Suite.runChild(Suite.java:27) org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)