Qpid
  1. Qpid
  2. QPID-3774

[Java Broker] Allow out of order completion of persistent enqueues / dequeues

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.14
    • Fix Version/s: 0.15
    • Component/s: Java Broker
    • Labels:
      None

      Description

      Currently each persistent enqueue or dequeue in a non transactional session is processed in strict order with a synchronous write occurring for each enqueue/dequeue.

      In order to improve performance we should make these enqueues/dequeues asynchronous and only wait for completion when a sync is requested.

      Note that due to the way the IO layer works, we cannot make this truly asynchronous as we need to synchronize before control is handed back to the IO layer as it then performs a blocking read. if the client is waiting for the completion from the broker it will never send more data on the wire, and since the read is blocking the broker cannot check to see if the async write to disk has completed. In practice tests have shown that this does not significantly change the maximum possible throughput.

        Activity

        Hide
        Keith Wall added a comment -

        Test case AcknowledgeTest was seen to fail occasionally on slower CI boxes. Committed additional change to test case to sync() after the subsequent message sends. (The original commit added this to the init path only).

        Show
        Keith Wall added a comment - Test case AcknowledgeTest was seen to fail occasionally on slower CI boxes. Committed additional change to test case to sync() after the subsequent message sends. (The original commit added this to the init path only).
        Hide
        Robbie Gemmell added a comment -

        The 0-8/0-9/0-9-1 path updates mentioned have also been made.

        Changes look reasonable to me.

        Show
        Robbie Gemmell added a comment - The 0-8/0-9/0-9-1 path updates mentioned have also been made. Changes look reasonable to me.
        Hide
        Rob Godfrey added a comment -

        Change made to 0-10 codepath to implement this optimisation. A similar change should be possible to the 0-8/0-9/0-9-1 codepath.

        the change only affects the BDB store (or other 3rd party stores that might implement asynchronous commits). The Derby store does not implement asynchronous commits and thus gains no advantage through this change.

        Show
        Rob Godfrey added a comment - Change made to 0-10 codepath to implement this optimisation. A similar change should be possible to the 0-8/0-9/0-9-1 codepath. the change only affects the BDB store (or other 3rd party stores that might implement asynchronous commits). The Derby store does not implement asynchronous commits and thus gains no advantage through this change.

          People

          • Assignee:
            Keith Wall
            Reporter:
            Rob Godfrey
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development