Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-8358

Deadlock in IgnitePdsAtomicCacheRebalancingTest

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.4
    • Fix Version/s: 2.5
    • Component/s: None
    • Labels:
      None

      Description

      Deadlocked threads are:

      [14:21:46] :     [Step 3/4] ##### DEADLOCKED Thread [name="sys-#22788%persistence.IgnitePdsAtomicCacheRebalancingTest2%", id=25953, state=WAITING, blockCnt=0, waitCnt=2]
      [14:21:46] :     [Step 3/4]     Lock [object=java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@adcfad9, ownerName=exchange-worker-#22778%persistence.IgnitePdsAtomicCacheRebalancingTest2%, ownerId=25941]
      [14:21:46] :     [Step 3/4]         at sun.misc.Unsafe.park(Native Method)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.localPartitionMap(GridDhtPartitionTopologyImpl.java:1000)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.GridCachePartitionExchangeManager.createPartitionsSingleMessage(GridCachePartitionExchangeManager.java:1250)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.GridCachePartitionExchangeManager.sendLocalPartitions(GridCachePartitionExchangeManager.java:1205)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.GridCachePartitionExchangeManager.refreshPartitions(GridCachePartitionExchangeManager.java:1036)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ResendTimeoutObject$1.run(GridCachePartitionExchangeManager.java:2663)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6751)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:110)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      [14:21:46] :     [Step 3/4]         at java.lang.Thread.run(Thread.java:745)
      [14:21:46] :     [Step 3/4]
      [14:21:46] :     [Step 3/4]     Locked synchronizers:
      [14:21:46] :     [Step 3/4]         java.util.concurrent.ThreadPoolExecutor$Worker@469d36ed
      
      [14:21:46] :     [Step 3/4] ##### DEADLOCKED Thread [name="sys-#22787%persistence.IgnitePdsAtomicCacheRebalancingTest2%", id=25952, state=WAITING, blockCnt=0, waitCnt=3]
      [14:21:46] :     [Step 3/4]     Lock [object=java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3a2e9f5b, ownerName=exchange-worker-#22778%persistence.IgnitePdsAtomicCacheRebalancingTest2%, ownerId=25941]
      [14:21:46] :     [Step 3/4]         at sun.misc.Unsafe.park(Native Method)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.StripedCompositeReadWriteLock$WriteLock.lock0(StripedCompositeReadWriteLock.java:154)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.StripedCompositeReadWriteLock$WriteLock.lock(StripedCompositeReadWriteLock.java:123)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.onEvicted(GridDhtPartitionTopologyImpl.java:2444)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPreloader.onPartitionEvicted(GridDhtPreloader.java:433)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition.finishDestroy(GridDhtLocalPartition.java:794)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition.finishEviction(GridDhtLocalPartition.java:775)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition.access$600(GridDhtLocalPartition.java:78)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition$ClearFuture.lambda$registerEvictionCallback$fb3468a7$1(GridDhtLocalPartition.java:1372)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition$ClearFuture$$Lambda$67/872465339.apply(Unknown Source)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:440)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition$ClearFuture.finish(GridDhtLocalPartition.java:1436)
      [14:21:46] :     [Step 3/4]         - locked o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition$ClearFuture@fb49fdf
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition.tryClear(GridDhtLocalPartition.java:861)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtPartitionsEvictor$1.call(GridDhtPartitionsEvictor.java:95)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtPartitionsEvictor$1.call(GridDhtPartitionsEvictor.java:73)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6695)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:110)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      [14:21:46] :     [Step 3/4]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      [14:21:46] :     [Step 3/4]         at java.lang.Thread.run(Thread.java:745)
      [14:21:46] :     [Step 3/4]
      [14:21:46] :     [Step 3/4]     Locked synchronizers:
      [14:21:46] :     [Step 3/4]         java.util.concurrent.ThreadPoolExecutor$Worker@33351244
      
      [14:21:46] :     [Step 3/4] ##### DEADLOCKED Thread [name="exchange-worker-#22778%persistence.IgnitePdsAtomicCacheRebalancingTest2%", id=25941, state=BLOCKED, blockCnt=1, waitCnt=694]
      [14:21:46] :     [Step 3/4]     Lock [object=o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition$ClearFuture@fb49fdf, ownerName=sys-#22787%persistence.IgnitePdsAtomicCacheRebalancingTest2%, ownerId=25952]
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition$ClearFuture.registerEvictionCallback(GridDhtLocalPartition.java:1362)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition$ClearFuture.initialize(GridDhtLocalPartition.java:1465)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition.clearAsync0(GridDhtLocalPartition.java:654)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtLocalPartition.rent(GridDhtLocalPartition.java:631)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.distributed.dht.GridDhtPartitionTopologyImpl.afterExchange(GridDhtPartitionTopologyImpl.java:735)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2488)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2299)
      [14:21:46] :     [Step 3/4]         at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:110)
      [14:21:46] :     [Step 3/4]         at java.lang.Thread.run(Thread.java:745)
      
      [14:21:46] :     [Step 3/4]     Locked synchronizers:
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@adcfad9
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@753fd7a1
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@12a2585b
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@21b6c9c2
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@309cedb6
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3b95a6db
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3bf54172
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@2c9a6717
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@7b3cde6f
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@6d091cad
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@7c663eaf
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3bb5ceb
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@1e692555
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3ba0ae41
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@76fe6cdc
      [14:21:46] :     [Step 3/4]         java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@3a2e9f5b
      
      

        Attachments

          Activity

            People

            • Assignee:
              Jokser Pavel Kovalenko
              Reporter:
              ascherbakov Alexei Scherbakov
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: