Uploaded image for project: 'ActiveMQ Artemis'
  1. ActiveMQ Artemis
  2. ARTEMIS-2276

Deadlock between QueueImpl and TransactionImpl

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.4
    • Fix Version/s: 2.7.0
    • Component/s: Broker
    • Labels:
      None

      Description

      Hi,

      we just experienced another deadlock in artemis.

      I am attaching the thread dump. The issue seems to be here:

       

      "Thread-0 (activemq-netty-threads)" Id=43 BLOCKED on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1 owned by "Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87
       at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1332)
       - blocked on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
       at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1328)
       at org.apache.activemq.artemis.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:1295)
       at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:207)
       at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:199)
       at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:874)
       - locked org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@31c578da
       at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:891)
       at org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:159)
       at org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:111)
       at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:469)
       at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2936)
       at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2921)
       - locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
      [...]
      "Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87 BLOCKED on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217 owned by "Thread-0 (activemq-netty-threads)" Id=43
       at org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:165)
       - blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
       at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546)
       - locked org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
      [...]
      

       

      Thread-0 locks QueueImpl@6b7ed217.
      Thread-7 locks TransactionImpl@196f00e1.
      Thread-0 waits for TransactionImpl@196f00e1.
      Thread-7 waits for QueueImpl@6b7ed217.

      We had a few mqtt producers, stomp consumers and stomp producers. A few hundred items/sec. deliverDirect disabled on mqtt as suggested by clebert suconic here: https://github.com/apache/activemq-artemis/pull/2577#issuecomment-472070556

      But I guess this dead lock is not the same as the one discussed there.

      Thanks for your help.

        Attachments

        1. artemis_thread_dump_2.log
          115 kB
          Frederik Petersen

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              FrederikP Frederik Petersen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: