[ARTEMIS-2603] Deadlock on testsuite: PagingStoreImpl::getCurrentIDs is a read operation

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.12.0
• Assignee: Clebert Suconic
• Reporter: Clebert Suconic
• Created: 17/Jan/20 16:04
• Updated: 16/Apr/20 22:52
• Resolved: 06/Feb/20 21:22

Description

This is fixing a deadlock found during a run on the testsuite.

This is because we acquire a writeLock on getCurrentIDs without being needed.
As we will always acquire a writeLock before writing a new file (opening a new page)

This would translate as a possible issue during startReplication if cleanup happens at the same time.

We turn of cleanup anyways to avoid this issue, but this would be an extra care for the issue not happening.

Java stack information for the threads listed above:
===================================================
"Thread-4895":
at sun.misc.Unsafe.park(Native Method)

• parking to wait for <0x00000007eff46968> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
  at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:943)
  at org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.getCurrentIds(PagingStoreImpl.java:1172)
  at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.getPageInformationForSync(JournalStorageManager.java:744)
  at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.startReplication(JournalStorageManager.java:672)
  at org.apache.activemq.artemis.core.server.impl.SharedNothingLiveActivation$2.run(SharedNothingLiveActivation.java:178)
  at java.lang.Thread.run(Thread.java:745)
  "Thread-0 (ActiveMQ-IO-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$7@58f73461)":
  at sun.misc.Unsafe.park(Native Method)
• parking to wait for <0x00000007f0101b20> (a java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
  at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
  at org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.readLock(AbstractJournalStorageManager.java:403)
  at org.apache.activemq.artemis.core.persistence.impl.journal.JournalStorageManager.pageClosed(JournalStorageManager.java:394)
  at org.apache.activemq.artemis.core.paging.impl.Page.close(Page.java:464)
• locked <0x00000007eff438e0> (a org.apache.activemq.artemis.core.paging.impl.Page)
  at org.apache.activemq.artemis.core.paging.impl.Page.close(Page.java:450)
  at org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.openNewPage(PagingStoreImpl.java:1114)
  at org.apache.activemq.artemis.core.paging.impl.PagingStoreImpl.forceAnotherPage(PagingStoreImpl.java:601)
  at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.cleanupComplete(PageCursorProviderImpl.java:514)
  at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl.cleanup(PageCursorProviderImpl.java:470)
• locked <0x00000007eff446f8> (a org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl)
  at org.apache.activemq.artemis.core.paging.cursor.impl.PageCursorProviderImpl$1.run(PageCursorProviderImpl.java:377)
  at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
  at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
  at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66)
  at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$2/1428475041.run(Unknown Source)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
  Found 1 deadlock.

Issue Links

• links to: GitHub Pull Request #2946