Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20188 [TESTING] Performance
  3. HBASE-20236

[locking] Write-time worst offenders

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-beta-2
    • Fix Version/s: None
    • Component/s: Performance
    • Labels:
      None

      Description

      Messing w/ my new toy, here are worst offenders locking; they must be bad if they show up in this sampling profiler:

       7 Total: 769321884622 (99.24%)  samples: 2965
        8   [ 0] java.util.concurrent.Semaphore$NonfairSync
        9   [ 1] sun.misc.Unsafe.park
       10   [ 2] java.util.concurrent.locks.LockSupport.park
       11   [ 3] java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
       12   [ 4] java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly
       13   [ 5] java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly
       14   [ 6] java.util.concurrent.Semaphore.acquire
       15   [ 7] org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner
       16   [ 8] org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run
       17
       18 Total: 4284274263 (0.55%)  samples: 23543
       19   [ 0] org.apache.hadoop.hbase.regionserver.MutableSegment
       20   [ 1] org.apache.hadoop.hbase.ByteBufferKeyValue.getSequenceId
       21   [ 2] org.apache.hadoop.hbase.regionserver.Segment.updateMetaInfo
       22   [ 3] org.apache.hadoop.hbase.regionserver.Segment.internalAdd
       23   [ 4] org.apache.hadoop.hbase.regionserver.MutableSegment.add
       24   [ 5] org.apache.hadoop.hbase.regionserver.AbstractMemStore.internalAdd
       25   [ 6] org.apache.hadoop.hbase.regionserver.AbstractMemStore.add
       26   [ 7] org.apache.hadoop.hbase.regionserver.AbstractMemStore.add
       27   [ 8] org.apache.hadoop.hbase.regionserver.HStore.add
       28   [ 9] org.apache.hadoop.hbase.regionserver.HRegion.applyToMemStore
       29   [10] org.apache.hadoop.hbase.regionserver.HRegion.access$600
       30   [11] org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.applyFamilyMapToMemStore
       31   [12] org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.lambda$writeMiniBatchOperationsToMemStore$0
       32   [13] org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation$$Lambda$442.1445825895.visit
       33   [14] org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.visitBatchOperations
       34   [15] org.apache.hadoop.hbase.regionserver.HRegion$BatchOperation.writeMiniBatchOperationsToMemStore
       35   [16] org.apache.hadoop.hbase.regionserver.HRegion$MutationBatchOperation.writeMiniBatchOperationsToMemStore
       36   [17] org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutate
       37   [18] org.apache.hadoop.hbase.regionserver.HRegion.batchMutate
       38   [19] org.apache.hadoop.hbase.regionserver.HRegion.batchMutate
       39   [20] org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp
       40   [21] org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicBatchOp
       41   [22] org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation
       42   [23] org.apache.hadoop.hbase.regionserver.RSRpcServices.multi
       43   [24] org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod
       44   [25] org.apache.hadoop.hbase.ipc.RpcServer.call
       45   [26] org.apache.hadoop.hbase.ipc.CallRunner.run
       46   [27] org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run
       47   [28] org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run
       48
       49 Total: 717708856 (0.09%)  samples: 214
       50   [ 0] java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync
       51   [ 1] sun.misc.Unsafe.park
       52   [ 2] java.util.concurrent.locks.LockSupport.park
       53   [ 3] java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt
       54   [ 4] java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued
       55   [ 5] java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire
       56   [ 6] java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock
       57   [ 7] org.apache.hadoop.hbase.regionserver.HRegion.blockUpdates
       58   [ 8] org.apache.hadoop.hbase.regionserver.RegionServicesForStores.blockUpdates
       59   [ 9] org.apache.hadoop.hbase.regionserver.CompactingMemStore.flushInMemory
       60   [10] org.apache.hadoop.hbase.regionserver.CompactingMemStore$InMemoryFlushRunnable.run
       61   [11] java.util.concurrent.ThreadPoolExecutor.runWorker
       62   [12] java.util.concurrent.ThreadPoolExecutor$Worker.run
       63   [13] java.lang.Thread.run
      ...
      

        Attachments

        1. 2.0623.111354.lock.svg
          158 kB
          stack
        2. HBASE-20236.master.001.patch
          26 kB
          stack
        3. no_semaphore_vs_semaphore.png
          29 kB
          stack

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stack stack
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: