Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9129

Scanner prefetching leaks scanners.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.95.1
    • Fix Version/s: 0.95.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Reverted Scanner Pre-fetching

      Description

      Running YCSB workload-e on 0.95 basically hangs the whole cluster.

      YCSB opens a scanner for 100 rows.

      1. YCSB Calls next
      2. Pre-fetching starts
      3. YCSB Closes scanner
      4. Pre-fetching re-adds the scanner.

      So the end result is:

      "scan-prefetch-2-thread-45" daemon prio=10 tid=0x00007f7e406ec800 nid=0x40bc runnable [0x00007f75ffefd000]
         java.lang.Thread.State: RUNNABLE
              at org.apache.hadoop.hbase.regionserver.Leases$Lease.equals(Leases.java:272)
              at java.util.PriorityQueue.indexOf(PriorityQueue.java:342)
              at java.util.PriorityQueue.remove(PriorityQueue.java:360)
              at java.util.concurrent.DelayQueue.remove(DelayQueue.java:476)
              at org.apache.hadoop.hbase.regionserver.Leases.removeLease(Leases.java:232)
              - locked <0x00007f774455a660> (a java.util.concurrent.DelayQueue)
              at org.apache.hadoop.hbase.regionserver.RegionScannerHolder$ScanPrefetcher.call(RegionScannerHolder.java:269)
              at org.apache.hadoop.hbase.regionserver.RegionScannerHolder$ScanPrefetcher.call(RegionScannerHolder.java:260)
              at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
              at java.util.concurrent.FutureTask.run(FutureTask.java:166)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:722)
      
      "scan-prefetch-2-thread-44" daemon prio=10 tid=0x00007f7e4c1ba800 nid=0x40bb waiting on condition [0x00007f7605b79000]
         java.lang.Thread.State: TIMED_WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x00007f774455ad78> (a java.util.concurrent.SynchronousQueue$TransferStack)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
              at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
              at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
              at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:722)
      
      "scan-prefetch-2-thread-43" daemon prio=10 tid=0x00007f7e38cbc800 nid=0x40ba waiting on condition [0x00007f7609ab8000]
         java.lang.Thread.State: TIMED_WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x00007f774455ad78> (a java.util.concurrent.SynchronousQueue$TransferStack)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
              at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
              at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
              at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
              at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:722)

        Attachments

        1. HBASE-9129-0.patch
          76 kB
          Elliott Neil Clark
        2. HBASE-9129-95-1.patch
          76 kB
          Elliott Neil Clark
        3. HBASE-9129-trunk-1.patch
          76 kB
          Elliott Neil Clark

          Activity

            People

            • Assignee:
              eclark Elliott Neil Clark
              Reporter:
              eclark Elliott Neil Clark
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: