Uploaded image for project: 'Qpid'
  1. Qpid
  2. QPID-2921

c++ broker: Improvements to asynchronos completion


    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8
    • Fix Version/s: 0.9
    • Component/s: C++ Broker
    • Labels:


        • Overview

      Asynchronous completion means that command execution is initiated in one thread
      (a client connection thread) and completed in a different thread.

      When the async store is loaded, message.transfer commands are
      completed by a store thread that does the async write.

        • Issues with asynchronous completion code as of revision r1029686
          • Not really asynchronous

      IncompleteMessageList::process blocks the connection thread till all
      outstanding async commands (messages) for the session are complete.

      With the new cluster, this could deadlock since it is blocking a Poller thread.

          • Race condition for message.transfer

      Sketch of the current code:

      // Called in connection thread

      { ++counter; }

      // Called in store thread once message is written.

      { if (--counter == 0) notifyCompleted(); }

      The intent is that notify be called once per message, after the
      message has been written to each queue it was routed to.

      However of a message is routed to N queues, it's possible for
      notifyCompleted to be called up to N times. The store thread could
      call notifyCompleted for the first queue before the connection thread
      has called enqueueAsync for the second queue, and so on.

          • No asynchronous completion for message.accept

      We do not currently delay completion of message.accept until the
      message is deleted from the async store. This could cause duplicate
      delivery if the broker crashes after completing the message but
      before it is removed from store.

      There is code in PersistableMessage to maintain a counter for dequeues
      analogous to to the async enqueue code but this is incorrect.

      Completion of transfer is triggered when all enqueues for a message are complete.
      Completion of accept is triggered for each dequeue from a queue independently.

      Furthermore a single accept can reference many messages, so it can't be associated with a message.

        • New requirements

      The new cluster design will need to participate in async completion, e.g.
      an accept cannot be comlpeted until the message is

      • removed from store (if present) AND
      • replicated to the cluster (if present) as dequeued

      The new cluster also needs to asynchronously complete binding commands
      (declare, bind, delete) when they are replicated to the cluster.


        1. proposal1.diff
          14 kB
          Ken Giusti
        2. msg.patch
          25 kB
          Ken Giusti

          Issue Links



              • Assignee:
                kgiusti Ken Giusti
                aconway Alan Conway
              • Votes:
                0 Vote for this issue
                1 Start watching this issue


                • Created: