Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
1.1.1, 1.3.0
-
None
-
None
-
None
Description
It happens on my production cluster when i run MR job. I save the dump.txt from this RS webUI.
Many threads stuck here:
Thread 133 (B.defaultRpcServer.handler=94,queue=4,port=16020): 32 State: WAITING 31 Blocked count: 477816 30 Waited count: 535255 29 Waiting on java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@6447ba67 28 Stack: 27 sun.misc.Unsafe.park(Native Method) 26 java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) 25 java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) 24 java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967) 23 java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283) 22 java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727) 21 org.apache.hadoop.hbase.regionserver.HStore.add(HStore.java:666) 20 org.apache.hadoop.hbase.regionserver.HRegion.applyFamilyMapToMemstore(HRegion.java:3621) 19 org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3038) 18 org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2793) 17 org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2735) 16 org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:692) 15 org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:654) 14 org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2029) 13 org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32213) 12 org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2112) 11 org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:101) 10 org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130) 9 org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107) 8 java.lang.Thread.run(Thread.java:745)
Attachments
Attachments
Issue Links
- is related to
-
HDFS-7005 DFS input streams do not timeout
- Closed