Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
0.95.1
-
None
-
None
-
Reviewed
-
Reverted Scanner Pre-fetching
Description
Running YCSB workload-e on 0.95 basically hangs the whole cluster.
YCSB opens a scanner for 100 rows.
- YCSB Calls next
- Pre-fetching starts
- YCSB Closes scanner
- Pre-fetching re-adds the scanner.
So the end result is:
"scan-prefetch-2-thread-45" daemon prio=10 tid=0x00007f7e406ec800 nid=0x40bc runnable [0x00007f75ffefd000] java.lang.Thread.State: RUNNABLE at org.apache.hadoop.hbase.regionserver.Leases$Lease.equals(Leases.java:272) at java.util.PriorityQueue.indexOf(PriorityQueue.java:342) at java.util.PriorityQueue.remove(PriorityQueue.java:360) at java.util.concurrent.DelayQueue.remove(DelayQueue.java:476) at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:232) - locked <0x00007f774455a660> (a java.util.concurrent.DelayQueue) at org.apache.hadoop.hbase.regionserver.RegionScannerHolder$ScanPrefetcher.call(RegionScannerHolder.java:269) at org.apache.hadoop.hbase.regionserver.RegionScannerHolder$ScanPrefetcher.call(RegionScannerHolder.java:260) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) "scan-prefetch-2-thread-44" daemon prio=10 tid=0x00007f7e4c1ba800 nid=0x40bb waiting on condition [0x00007f7605b79000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f774455ad78> (a java.util.concurrent.SynchronousQueue$TransferStack) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722) "scan-prefetch-2-thread-43" daemon prio=10 tid=0x00007f7e38cbc800 nid=0x40ba waiting on condition [0x00007f7609ab8000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00007f774455ad78> (a java.util.concurrent.SynchronousQueue$TransferStack) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)