Qpid
  1. Qpid
  2. QPID-3774

[Java Broker] Allow out of order completion of persistent enqueues / dequeues

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.14
    • Fix Version/s: 0.15
    • Component/s: Java Broker
    • Labels:
      None

      Description

      Currently each persistent enqueue or dequeue in a non transactional session is processed in strict order with a synchronous write occurring for each enqueue/dequeue.

      In order to improve performance we should make these enqueues/dequeues asynchronous and only wait for completion when a sync is requested.

      Note that due to the way the IO layer works, we cannot make this truly asynchronous as we need to synchronize before control is handed back to the IO layer as it then performs a blocking read. if the client is waiting for the completion from the broker it will never send more data on the wire, and since the read is blocking the broker cannot check to see if the async write to disk has completed. In practice tests have shown that this does not significantly change the maximum possible throughput.

        Activity

        Rob Godfrey created issue -
        Hide
        Rob Godfrey added a comment -

        Change made to 0-10 codepath to implement this optimisation. A similar change should be possible to the 0-8/0-9/0-9-1 codepath.

        the change only affects the BDB store (or other 3rd party stores that might implement asynchronous commits). The Derby store does not implement asynchronous commits and thus gains no advantage through this change.

        Show
        Rob Godfrey added a comment - Change made to 0-10 codepath to implement this optimisation. A similar change should be possible to the 0-8/0-9/0-9-1 codepath. the change only affects the BDB store (or other 3rd party stores that might implement asynchronous commits). The Derby store does not implement asynchronous commits and thus gains no advantage through this change.
        Rob Godfrey made changes -
        Field Original Value New Value
        Status Open [ 1 ] In Progress [ 3 ]
        Rob Godfrey made changes -
        Status In Progress [ 3 ] Ready To Review [ 10006 ]
        Hide
        Robbie Gemmell added a comment -

        The 0-8/0-9/0-9-1 path updates mentioned have also been made.

        Changes look reasonable to me.

        Show
        Robbie Gemmell added a comment - The 0-8/0-9/0-9-1 path updates mentioned have also been made. Changes look reasonable to me.
        Robbie Gemmell made changes -
        Status Ready To Review [ 10006 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Keith Wall added a comment -

        Test case AcknowledgeTest was seen to fail occasionally on slower CI boxes. Committed additional change to test case to sync() after the subsequent message sends. (The original commit added this to the init path only).

        Show
        Keith Wall added a comment - Test case AcknowledgeTest was seen to fail occasionally on slower CI boxes. Committed additional change to test case to sync() after the subsequent message sends. (The original commit added this to the init path only).
        Keith Wall made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Assignee Rob Godfrey [ rgodfrey ] Keith Wall [ k-wall ]
        Keith Wall made changes -
        Status Reopened [ 4 ] Closed [ 6 ]
        Resolution Fixed [ 1 ]
        Rob Godfrey made changes -
        Component/s Java Broker BDB Store [ 12315809 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        5d 14h 1 Rob Godfrey 26/Jan/12 10:27
        In Progress In Progress Reviewable Reviewable
        3s 1 Rob Godfrey 26/Jan/12 10:27
        Reviewable Reviewable Resolved Resolved
        3m 36s 1 Robbie Gemmell 26/Jan/12 10:30
        Resolved Resolved Reopened Reopened
        32d 3h 38m 1 Keith Wall 27/Feb/12 14:09
        Reopened Reopened Closed Closed
        17s 1 Keith Wall 27/Feb/12 14:09

          People

          • Assignee:
            Keith Wall
            Reporter:
            Rob Godfrey
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development