Uploaded image for project: 'Qpid'
  1. Qpid
  2. QPID-6374

[Java Client] Eliminate deadlocks by acquiring locks in a consistent order

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.31
    • Component/s: JMS AMQP 0-x
    • Labels:
      None

      Description

      A number of JIRAs exist relating to deadlocks in the Java client.

      Deadlocks can generally be avoided if locks are always acquired in the same order however the Java client threading model can make this hard.

      Two locks which in combination cause particular issues are the messageDeliveryLock and the failoverMutex.

      The semantic purpose of the messageDeliveryLock is to meet the requirements of JMS that session.close(), connection.close() and consumer.close() block while the application is receiving a message in onMessage() or from receive(). The lock should only be acquired from an application thread or from the onMessage dispatcher thread - it should not be held by a thread acting on behalf of processing caused by io processing (e.g. the broker closing the session or connection). The lock is session specific.

      The semantic purpose of the failover mutex is to ensure that application actions which cause state change at the broker can are not engaged in while failover is occurring. The lock is connection wide. In general the lock should only be held by application threads, except for when failover is occurring, in which case the thread managing failover will hold the lock.

      Since messageDeliveryLock has a narrower scope (session) than the failoverMutex (which is connection scoped) it makes sense to ensure that the messageDeliveryLock is always acquired first. Changing this ordering where necessary, and removing and path where the messageDeliveryLock can be acquired by a non-application thread will greatly reduce the number of deadlocks seen.

      Other deadlocks can be eliminated by removing the failoverMutex being acquired by threads acting on behalf of io processing, and by only holding the sessionCreationLock when setting the closed state or when actually creating a session.

        Attachments

        1. client_deadlock_3.diff
          12 kB
          Robert Godfrey

          Issue Links

            Activity

              People

              • Assignee:
                rgodfrey Robert Godfrey
                Reporter:
                rgodfrey Robert Godfrey
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: