Camel
  1. Camel
  2. CAMEL-5390

Option to assign unique correlation ID to JMS messages

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.10.0
    • Fix Version/s: 2.9.5, 2.10.3, 2.11.0
    • Component/s: camel-activemq
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      Imagine a sequential JMS invocation across Camel routes, such as the following:

      <route>
          <from uri="timer:foo?fixedRate=true&amp;period=10000" />
          <inOut uri="activemq:queue:test1" />
          <inOut uri="activemq:queue:test2" />
      </route>
      

      The camel-jms consumer listening on test1 will set the JMSCorrelationID header to the JMSMessageID for that exchange.

      When the response returns to the route and sent to test2, this consumer also uses the same JMSCorrelationID as before.

      IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

      Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:

              <route>
                  <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                  <setBody><constant>1,2,3,4,5</constant></setBody>
                  <inOut uri="activemq:queue:test1" />
                  <split parallelProcessing="true">
                      <tokenize token="," />
                      <inOut uri="activemq:queue:test2" />
                  </split> 
                  <to uri="log:Finished?showAll=true" />
              </route>
              
              <route>
                  <from uri="activemq:queue:test1" />
                  <to uri="log:Received?showAll=true" />
              </route>
       
               <route>
                  <from uri="activemq:queue:test2" />
                  <to uri="log:Received?showAll=true" />
                  <setBody><constant>reply</constant></setBody>
                  <delay><constant>100</constant></delay>
              </route>
      

      There are several solutions here:

      1. Remove the JMSCorrelationID header before each request to test2
      2. Enable the useMessageIDAsCorrelationID option on both producer and consumer endpoints
      3. Create a new option to assign a unique ID to the JMSCorrelationID header

      The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me this is inefficient and flaky for enterprise deployments.

      That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

      I propose calling this option 'assignUniqueCorrelationID'.

      I'm happy to submit a patch for this, targeting 2.10.1.

        Activity

        Hide
        Claus Ibsen added a comment -

        Ad 1)
        Ideally if Camel auto assign a JMSCorrelationID, then that id could be removed when the reply comes back, as its not an explicit configured correlation id done by the end user

        Ad 2)
        Yeah this works in the sense the broker assigns a new ID for each message send to it, and don't reuse a previous message id.
        The caveat is that the JMSMessageID is only available to Camel after the message has been sent, so we have extended logic to deal with that.

        Ad 3)
        This is what Camel does today by default, auto assign unique correlation ID, if no JMSCorrelationID is preset.
        Either we can do as #1, or we can introduce a new option to force creating a new unique ID even if JMSCorrelationID already exists. Then ppl can configure this on the component level to have it in use always.

        Option #1 is most compelling as its transparent and no ppl need to enable any option etc. The only downside is that the JMSCorrelationID header would be removed when the reply message is processed, where as before it was present.

        4)
        A new solution could be.

        We could prefix the generated JMSCorrelationID (from #1) by Camel or Auto or Generated etc. eg some token, so we know its an auto generated ID, and therefor the producer can "detect" this and then re-assign a new auto generated id. Then all together this is transparent for the end users, and no changes whatsoever is needed at all.

        Show
        Claus Ibsen added a comment - Ad 1) Ideally if Camel auto assign a JMSCorrelationID, then that id could be removed when the reply comes back, as its not an explicit configured correlation id done by the end user Ad 2) Yeah this works in the sense the broker assigns a new ID for each message send to it, and don't reuse a previous message id. The caveat is that the JMSMessageID is only available to Camel after the message has been sent, so we have extended logic to deal with that. Ad 3) This is what Camel does today by default, auto assign unique correlation ID, if no JMSCorrelationID is preset. Either we can do as #1, or we can introduce a new option to force creating a new unique ID even if JMSCorrelationID already exists. Then ppl can configure this on the component level to have it in use always. Option #1 is most compelling as its transparent and no ppl need to enable any option etc. The only downside is that the JMSCorrelationID header would be removed when the reply message is processed, where as before it was present. 4) A new solution could be. We could prefix the generated JMSCorrelationID (from #1) by Camel or Auto or Generated etc. eg some token, so we know its an auto generated ID, and therefor the producer can "detect" this and then re-assign a new auto generated id. Then all together this is transparent for the end users, and no changes whatsoever is needed at all.
        Hide
        Raúl Kripalani added a comment -

        Yes, 4) should work too. Could there be a reason why a user would want to reuse an old JMSCorrelationID? Just saying because by introducing #4 we alter past behaviour unless we introduce an option flag to negate/enable the correlation ID re-generation. In that case, I would prefer to be the least intrusive and go with #3, i.e. an option to force a generation of a new, explicit JMSCorrelationID with every send.

        Show
        Raúl Kripalani added a comment - Yes, 4) should work too. Could there be a reason why a user would want to reuse an old JMSCorrelationID? Just saying because by introducing #4 we alter past behaviour unless we introduce an option flag to negate/enable the correlation ID re-generation. In that case, I would prefer to be the least intrusive and go with #3, i.e. an option to force a generation of a new, explicit JMSCorrelationID with every send.
        Hide
        Raúl Kripalani added a comment -

        Claus, different day, fresh view. I think I misunderstood your proposal #4, now it makes more sense! I think this will be my first commit at Camel

        Show
        Raúl Kripalani added a comment - Claus, different day, fresh view. I think I misunderstood your proposal #4, now it makes more sense! I think this will be my first commit at Camel
        Hide
        Raúl Kripalani added a comment -

        Fixed in r1387808.

        Show
        Raúl Kripalani added a comment - Fixed in r1387808.
        Hide
        Raúl Kripalani added a comment -

        New behaviour of JmsProducer: The "Camel-" prefix is prepended to the generated JMSCorrelationId. The key is regenerated with every send when a previous JMSCorrelationId exists and it begins with "Camel-".

        Added two tests: JmsInOutParallelTest which represents the route in the ticket description and JmsInOutRepeatedInvocationsTest to test sequential sends to the same JMS endpoint.

        Show
        Raúl Kripalani added a comment - New behaviour of JmsProducer: The "Camel-" prefix is prepended to the generated JMSCorrelationId. The key is regenerated with every send when a previous JMSCorrelationId exists and it begins with "Camel-". Added two tests: JmsInOutParallelTest which represents the route in the ticket description and JmsInOutRepeatedInvocationsTest to test sequential sends to the same JMS endpoint.
        Hide
        Claus Ibsen added a comment -

        Since this is transparent I would like to backport this to 2.10 as well, as its being included in the SMX 4.5.x releases.

        Show
        Claus Ibsen added a comment - Since this is transparent I would like to backport this to 2.10 as well, as its being included in the SMX 4.5.x releases.

          People

          • Assignee:
            Raúl Kripalani
            Reporter:
            Raúl Kripalani
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development