Qpid
  1. Qpid
  2. QPID-4997

Dispatch - Thread safety issues in the usage of Proton

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Future
    • Component/s: Qpid Dispatch
    • Labels:
      None

      Description

      Dispatch, being an intermediary, often needs to deal with activating and modifying multiple links synchronously. There are several Proton calls that are invoked on threads in violation of the one-thread-at-a-time-per-connection rule.

      A mechanism needs to be put in place whereby delivery (settlement/disposition) and link (flow/offer) can be safely scheduled for later processing on a "proper" thread.

        Activity

        Ted Ross created issue -
        Hide
        Ted Ross added a comment -

        Rather than attempt to make the Proton objects thread-safe, the approach to fixing this problem is to enqueue not just messages for delivery on a different connection but also state change events (settlement, disposition change, etc.).

        Previously, message deliveries were enqueued for delivery on a different thread but disposition changes, settlement, and offered were done synchronously in the receiving thread.

        As changed, all of the above are enqueued for asynchronous processing in a thread that has the proper access to the target connection.

        Show
        Ted Ross added a comment - Rather than attempt to make the Proton objects thread-safe, the approach to fixing this problem is to enqueue not just messages for delivery on a different connection but also state change events (settlement, disposition change, etc.). Previously, message deliveries were enqueued for delivery on a different thread but disposition changes, settlement, and offered were done synchronously in the receiving thread. As changed, all of the above are enqueued for asynchronous processing in a thread that has the proper access to the target connection.
        Hide
        ASF subversion and git services added a comment -

        Commit 1511021 from Ted Ross in branch 'qpid/trunk'
        [ https://svn.apache.org/r1511021 ]

        QPID-5045 - Refactored the router data structures to support message and link routing.
        QPID-4997 - Fixed the thread safety problem.

        • Wrapped "pn_delivery" in the container to allow for explicit linkage of delivery pairs.
        • Removed the linkage between "message" and "delivery". Messages can now be discarded as
          soon as the last copy is delivered because their status will be tracked in the dx_delivery
          object.
        • Removed tx_handler from the set of callbacks in the container. There is no need for this
          notification (sendable delivery) because Dispatch does not create outbound deliveries
          until it is ready to send them (i.e. deliveries are created and advanced in one synchronous
          operation).
        • Replaced the out_fifo of messages per outbound-link with a pair of fifos (one for messages
          and one for state changes) per link. Note that even inbound-links need to send state
          changes outbound. This change addresses QPID-4997.
        Show
        ASF subversion and git services added a comment - Commit 1511021 from Ted Ross in branch 'qpid/trunk' [ https://svn.apache.org/r1511021 ] QPID-5045 - Refactored the router data structures to support message and link routing. QPID-4997 - Fixed the thread safety problem. Wrapped "pn_delivery" in the container to allow for explicit linkage of delivery pairs. Removed the linkage between "message" and "delivery". Messages can now be discarded as soon as the last copy is delivered because their status will be tracked in the dx_delivery object. Removed tx_handler from the set of callbacks in the container. There is no need for this notification (sendable delivery) because Dispatch does not create outbound deliveries until it is ready to send them (i.e. deliveries are created and advanced in one synchronous operation). Replaced the out_fifo of messages per outbound-link with a pair of fifos (one for messages and one for state changes) per link. Note that even inbound-links need to send state changes outbound. This change addresses QPID-4997 .
        Hide
        ASF subversion and git services added a comment -

        Commit 1511021 from Ted Ross in branch 'qpid/trunk'
        [ https://svn.apache.org/r1511021 ]

        QPID-5045 - Refactored the router data structures to support message and link routing.
        QPID-4997 - Fixed the thread safety problem.

        • Wrapped "pn_delivery" in the container to allow for explicit linkage of delivery pairs.
        • Removed the linkage between "message" and "delivery". Messages can now be discarded as
          soon as the last copy is delivered because their status will be tracked in the dx_delivery
          object.
        • Removed tx_handler from the set of callbacks in the container. There is no need for this
          notification (sendable delivery) because Dispatch does not create outbound deliveries
          until it is ready to send them (i.e. deliveries are created and advanced in one synchronous
          operation).
        • Replaced the out_fifo of messages per outbound-link with a pair of fifos (one for messages
          and one for state changes) per link. Note that even inbound-links need to send state
          changes outbound. This change addresses QPID-4997.
        Show
        ASF subversion and git services added a comment - Commit 1511021 from Ted Ross in branch 'qpid/trunk' [ https://svn.apache.org/r1511021 ] QPID-5045 - Refactored the router data structures to support message and link routing. QPID-4997 - Fixed the thread safety problem. Wrapped "pn_delivery" in the container to allow for explicit linkage of delivery pairs. Removed the linkage between "message" and "delivery". Messages can now be discarded as soon as the last copy is delivered because their status will be tracked in the dx_delivery object. Removed tx_handler from the set of callbacks in the container. There is no need for this notification (sendable delivery) because Dispatch does not create outbound deliveries until it is ready to send them (i.e. deliveries are created and advanced in one synchronous operation). Replaced the out_fifo of messages per outbound-link with a pair of fifos (one for messages and one for state changes) per link. Note that even inbound-links need to send state changes outbound. This change addresses QPID-4997 .
        Ted Ross made changes -
        Field Original Value New Value
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s Future [ 12315490 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Ted Ross
            Reporter:
            Ted Ross
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development