Uploaded image for project: 'Qpid Dispatch'
  1. Qpid Dispatch
  2. DISPATCH-669

Dispatch - Crash occurs when linked deliveries are concurrently settled

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Dispatch router crashed while running a test in which a client/server pair exchanges requests and responses with acknowledgement (not pre-settled). The test does not throttle the message rates.

      After several tens-of-thousands of transfers, the router crashed while settling a delivery and attempting to unlink it from its peer. Apparently two linked deliveries were concurrently settled on separate threads.

        Activity

        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1548209 from Ted Ross in branch 'dispatch/trunk'
        [ https://svn.apache.org/r1548209 ]

        QPID-5397 - Added a global lock to protect the linking and unlinking of peered deliveries.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1548209 from Ted Ross in branch 'dispatch/trunk' [ https://svn.apache.org/r1548209 ] QPID-5397 - Added a global lock to protect the linking and unlinking of peered deliveries.
        Hide
        tedross Ted Ross added a comment -

        The fix was an improvement, but there is still a window of opportunity for a problem.

        If a delivery is sent unsettled, and is then settled by the sender before being settled by the receiver (due to exceeding the outgoing window in Messenger for example), Dispatch may attempt to re-settle the sender-side delivery when the receiver settles thus causing problems in Proton Engine and possibly double-freeing the structure.

        A new fix will hold off the unlinking and freeing of linked deliveries until they are both settled.

        Show
        tedross Ted Ross added a comment - The fix was an improvement, but there is still a window of opportunity for a problem. If a delivery is sent unsettled, and is then settled by the sender before being settled by the receiver (due to exceeding the outgoing window in Messenger for example), Dispatch may attempt to re-settle the sender-side delivery when the receiver settles thus causing problems in Proton Engine and possibly double-freeing the structure. A new fix will hold off the unlinking and freeing of linked deliveries until they are both settled.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1548737 from Ted Ross in branch 'dispatch/trunk'
        [ https://svn.apache.org/r1548737 ]

        QPID-5397

        • Redesigned the fix to the mutex problem
        • Added a test to detect the settlement-collision problem
        • Increased the verbosity of the system tests in qdtest
        • Some minor cleanup in the python router and qdstat
        Show
        jira-bot ASF subversion and git services added a comment - Commit 1548737 from Ted Ross in branch 'dispatch/trunk' [ https://svn.apache.org/r1548737 ] QPID-5397 Redesigned the fix to the mutex problem Added a test to detect the settlement-collision problem Increased the verbosity of the system tests in qdtest Some minor cleanup in the python router and qdstat

          People

          • Assignee:
            tedross Ted Ross
            Reporter:
            tedross Ted Ross
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development