Uploaded image for project: 'Qpid'
  1. Qpid
  2. QPID-3774

[Java Broker] Allow out of order completion of persistent enqueues / dequeues

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.14
    • Fix Version/s: 0.15
    • Component/s: Java Broker
    • Labels:
      None

      Description

      Currently each persistent enqueue or dequeue in a non transactional session is processed in strict order with a synchronous write occurring for each enqueue/dequeue.

      In order to improve performance we should make these enqueues/dequeues asynchronous and only wait for completion when a sync is requested.

      Note that due to the way the IO layer works, we cannot make this truly asynchronous as we need to synchronize before control is handed back to the IO layer as it then performs a blocking read. if the client is waiting for the completion from the broker it will never send more data on the wire, and since the read is blocking the broker cannot check to see if the async write to disk has completed. In practice tests have shown that this does not significantly change the maximum possible throughput.

        Activity

        Hide
        rgodfrey Rob Godfrey added a comment -

        Change made to 0-10 codepath to implement this optimisation. A similar change should be possible to the 0-8/0-9/0-9-1 codepath.

        the change only affects the BDB store (or other 3rd party stores that might implement asynchronous commits). The Derby store does not implement asynchronous commits and thus gains no advantage through this change.

        Show
        rgodfrey Rob Godfrey added a comment - Change made to 0-10 codepath to implement this optimisation. A similar change should be possible to the 0-8/0-9/0-9-1 codepath. the change only affects the BDB store (or other 3rd party stores that might implement asynchronous commits). The Derby store does not implement asynchronous commits and thus gains no advantage through this change.
        Hide
        gemmellr Robbie Gemmell added a comment -

        The 0-8/0-9/0-9-1 path updates mentioned have also been made.

        Changes look reasonable to me.

        Show
        gemmellr Robbie Gemmell added a comment - The 0-8/0-9/0-9-1 path updates mentioned have also been made. Changes look reasonable to me.
        Hide
        k-wall Keith Wall added a comment -

        Test case AcknowledgeTest was seen to fail occasionally on slower CI boxes. Committed additional change to test case to sync() after the subsequent message sends. (The original commit added this to the init path only).

        Show
        k-wall Keith Wall added a comment - Test case AcknowledgeTest was seen to fail occasionally on slower CI boxes. Committed additional change to test case to sync() after the subsequent message sends. (The original commit added this to the init path only).

          People

          • Assignee:
            k-wall Keith Wall
            Reporter:
            rgodfrey Rob Godfrey
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development