Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-13003

RabbitMQ - Publisher confirms kills performance

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 2.23.0
    • Future
    • camel-rabbitmq
    • None
    • Unknown

    Description

      Hi Camelers,

      Disclaimer: I'm fairly new to Camel, so please bear with me.

      I would like to discuss https://github.com/apache/camel/pull/717 and possibly propose an improvement.

      In the PR mentioned above, support for publisher confirms has been implemented. The related documentation can be found on https://www.rabbitmq.com/confirms.html#publisher-confirms.

      My thoughts is that the current implementation does not perform well. Basically, it has a throughput of one message at a time. I'll try to explain.

      I believe this means that our throughput is limited to one message at a time. For each message sent to RabbitMQ, we have to await the confirmation.

      I believe it would be better to avoid using `channel.waitForConfirmsOrDie()` and implement a `ConfirmListener` ourselves. See https://github.com/rabbitmq/rabbitmq-java-client/blob/master/src/main/java/com/rabbitmq/client/ConfirmListener.java. This way, we can continue publications of subsequent messages, while we receive ACK's or NACK's of earlier publications.

      An implementation gotcha is the fact that upon broker restart/disconnection the callback `ConfirmListener.nack()` will not be automatically called by the RabbitMQ, as the RabbitMQ API does not keep track of the in-flight messages. (In case you are wondering, `channel.waitForConfirmsOrDie()` is implemented by counting the responses.) In other words, we will have to assume a NACK for all in-flight messages when the connection was closed. This also means that we have to keep track of all in-flight messages, together with the corresponding `AsyncCallback`, at our end.

      We also might want to limit the number of in-flight messages between the component and the broker.

      What do you think?

      Attachments

        Activity

          People

            Unassigned Unassigned
            pbillen Peter Billen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: