Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-13003

RabbitMQ - Publisher confirms kills performance

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.23.0
    • Fix Version/s: Future
    • Component/s: camel-rabbitmq
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      Hi Camelers,

      Disclaimer: I'm fairly new to Camel, so please bear with me.

      I would like to discuss https://github.com/apache/camel/pull/717 and possibly propose an improvement.

      In the PR mentioned above, support for publisher confirms has been implemented. The related documentation can be found on https://www.rabbitmq.com/confirms.html#publisher-confirms.

      My thoughts is that the current implementation does not perform well. Basically, it has a throughput of one message at a time. I'll try to explain.

      I believe this means that our throughput is limited to one message at a time. For each message sent to RabbitMQ, we have to await the confirmation.

      I believe it would be better to avoid using `channel.waitForConfirmsOrDie()` and implement a `ConfirmListener` ourselves. See https://github.com/rabbitmq/rabbitmq-java-client/blob/master/src/main/java/com/rabbitmq/client/ConfirmListener.java. This way, we can continue publications of subsequent messages, while we receive ACK's or NACK's of earlier publications.

      An implementation gotcha is the fact that upon broker restart/disconnection the callback `ConfirmListener.nack()` will not be automatically called by the RabbitMQ, as the RabbitMQ API does not keep track of the in-flight messages. (In case you are wondering, `channel.waitForConfirmsOrDie()` is implemented by counting the responses.) In other words, we will have to assume a NACK for all in-flight messages when the connection was closed. This also means that we have to keep track of all in-flight messages, together with the corresponding `AsyncCallback`, at our end.

      We also might want to limit the number of in-flight messages between the component and the broker.

      What do you think?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              pbillen Peter Billen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: