Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-5527

GridDeferredAckMessageSender can cause starvation in stripe pool on unstable topology.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1
    • Component/s: clients, general
    • Labels:
      None

      Description

      Slow or failed client can cause starvation in stripe-pool.

      GridDeferredAckMessageSender.sendDeferredAckMessage() can stucks when add message to buffer while other thread trying to send this buffer.

      Seems, we shouldn't wait for readLock in DeferredAckMessageBuffer.add() method and return false immediately.

      Stacktrace:

      Thread name="sys-stripe-3-#4%null%", id=16, state=WAITING, blockCnt=3087, waitCnt=204846
      Lock [object=java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@54ef60c9, ownerName=sys-#12106%null%, ownerId=12326]
      at sun.misc.Unsafe.park(Native Method)
      at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
      at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
      at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
      at o.a.i.i.processors.cache.GridDeferredAckMessageSender$DeferredAckMessageBuffer.add(GridDeferredAckMessageSender.java:176)
      at o.a.i.i.processors.cache.GridDeferredAckMessageSender.sendDeferredAckMessage(GridDeferredAckMessageSender.java:100)
      at o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.sendDeferredUpdateResponse(GridDhtAtomicCache.java:3371)
      at o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.processDhtAtomicUpdateRequest(GridDhtAtomicCache.java:3353)
      at o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$1800(GridDhtAtomicCache.java:130)
      at o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$8.apply(GridDhtAtomicCache.java:379)
      at o.a.i.i.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$8.apply(GridDhtAtomicCache.java:374)
      at o.a.i.i.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:827)
      at o.a.i.i.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:369)
      at o.a.i.i.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:293)
      at o.a.i.i.processors.cache.GridCacheIoManager.access$000(GridCacheIoManager.java:95)
      at o.a.i.i.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:238)
      at o.a.i.i.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1217)
      at o.a.i.i.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:845)
      at o.a.i.i.managers.communication.GridIoManager.access$2100(GridIoManager.java:109)
      at o.a.i.i.managers.communication.GridIoManager$6.run(GridIoManager.java:785)
      at o.a.i.i.util.StripedExecutor$Stripe.run(StripedExecutor.java:428)
      at java.lang.Thread.run(Thread.java:745)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                amashenkov Andrey Mashenkov
                Reporter:
                amashenkov Andrey Mashenkov
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: