Camel
  1. Camel
  2. CAMEL-5390

Option to assign unique correlation ID to JMS messages

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.10.0
    • Fix Version/s: 2.9.5, 2.10.3, 2.11.0
    • Component/s: camel-activemq
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      Imagine a sequential JMS invocation across Camel routes, such as the following:

      <route>
          <from uri="timer:foo?fixedRate=true&amp;period=10000" />
          <inOut uri="activemq:queue:test1" />
          <inOut uri="activemq:queue:test2" />
      </route>
      

      The camel-jms consumer listening on test1 will set the JMSCorrelationID header to the JMSMessageID for that exchange.

      When the response returns to the route and sent to test2, this consumer also uses the same JMSCorrelationID as before.

      IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

      Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:

              <route>
                  <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                  <setBody><constant>1,2,3,4,5</constant></setBody>
                  <inOut uri="activemq:queue:test1" />
                  <split parallelProcessing="true">
                      <tokenize token="," />
                      <inOut uri="activemq:queue:test2" />
                  </split> 
                  <to uri="log:Finished?showAll=true" />
              </route>
              
              <route>
                  <from uri="activemq:queue:test1" />
                  <to uri="log:Received?showAll=true" />
              </route>
       
               <route>
                  <from uri="activemq:queue:test2" />
                  <to uri="log:Received?showAll=true" />
                  <setBody><constant>reply</constant></setBody>
                  <delay><constant>100</constant></delay>
              </route>
      

      There are several solutions here:

      1. Remove the JMSCorrelationID header before each request to test2
      2. Enable the useMessageIDAsCorrelationID option on both producer and consumer endpoints
      3. Create a new option to assign a unique ID to the JMSCorrelationID header

      The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me this is inefficient and flaky for enterprise deployments.

      That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

      I propose calling this option 'assignUniqueCorrelationID'.

      I'm happy to submit a patch for this, targeting 2.10.1.

        Activity

        Claus Ibsen made changes -
        Fix Version/s 2.9.5 [ 12323363 ]
        Claus Ibsen made changes -
        Fix Version/s 2.10.3 [ 12323366 ]
        Raúl Kripalani made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Fix Version/s 2.11.0 [ 12321695 ]
        Resolution Fixed [ 1 ]
        Raúl Kripalani made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Daniel Kulp made changes -
        Fix Version/s 2.10.1 [ 12321259 ]
        Raúl Kripalani made changes -
        Description Imagine a sequential JMS invocation across Camel routes, such as the following:

        {code}
        <route>
            <from uri="timer:foo?fixedRate=true&amp;period=10000" />
            <inOut uri="activemq:queue:test1" />
            <inOut uri="activemq:queue:test2" />
        </route>
        {code}

        The camel-jms consumer listening on test1 will set the {{JMSCorrelationID}} header to the {{JMSMessageID}} for that exchange.

        When the response returns to the route and sent to test2, this consumer also uses the *same* {{JMSCorrelationID}} as before.

        IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

        Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:
        \\
        {code}
                <route>
                    <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                    <setBody><constant>1,2,3,4,5</constant></setBody>
                    <inOut uri="activemq:queue:test1" />
                    <split parallelProcessing="true">
                        <tokenize token="," />
                        <inOut uri="activemq:queue:test2" />
                    </split>
                    <to uri="log:Finished?showAll=true" />
                </route>
                
                <route>
                    <from uri="activemq:queue:test1" />
                    <to uri="log:Received?showAll=true" />
                </route>
         
                 <route>
                    <from uri="activemq:queue:test2" />
                    <to uri="log:Received?showAll=true" />
                    <setBody><constant>reply</constant></setBody>
                    <delay><constant>100</constant></delay>
                </route>
        {code}

        There are several solutions here:

        # Remove the JMSCorrelationID header before each request to test2
        # Enable the useMessageIDAsCorrelationID option to true on both producer and consumer
        # Create a new option to assign a unique ID to the JMSCorrelationID header

        The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me this is inefficient and flaky for enterprise deployments.

        That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

        I propose calling this option 'assignUniqueCorrelationID'.

        I'm happy to submit a patch for this, targeting 2.10.1.
        Imagine a sequential JMS invocation across Camel routes, such as the following:

        {code}
        <route>
            <from uri="timer:foo?fixedRate=true&amp;period=10000" />
            <inOut uri="activemq:queue:test1" />
            <inOut uri="activemq:queue:test2" />
        </route>
        {code}

        The camel-jms consumer listening on test1 will set the {{JMSCorrelationID}} header to the {{JMSMessageID}} for that exchange.

        When the response returns to the route and sent to test2, this consumer also uses the *same* {{JMSCorrelationID}} as before.

        IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

        Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:
        \\
        {code}
                <route>
                    <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                    <setBody><constant>1,2,3,4,5</constant></setBody>
                    <inOut uri="activemq:queue:test1" />
                    <split parallelProcessing="true">
                        <tokenize token="," />
                        <inOut uri="activemq:queue:test2" />
                    </split>
                    <to uri="log:Finished?showAll=true" />
                </route>
                
                <route>
                    <from uri="activemq:queue:test1" />
                    <to uri="log:Received?showAll=true" />
                </route>
         
                 <route>
                    <from uri="activemq:queue:test2" />
                    <to uri="log:Received?showAll=true" />
                    <setBody><constant>reply</constant></setBody>
                    <delay><constant>100</constant></delay>
                </route>
        {code}

        There are several solutions here:

        # Remove the JMSCorrelationID header before each request to test2
        # Enable the useMessageIDAsCorrelationID option on both producer and consumer endpoints
        # Create a new option to assign a unique ID to the JMSCorrelationID header

        The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me this is inefficient and flaky for enterprise deployments.

        That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

        I propose calling this option 'assignUniqueCorrelationID'.

        I'm happy to submit a patch for this, targeting 2.10.1.
        Raúl Kripalani made changes -
        Description Imagine a sequential JMS invocation across Camel routes, such as the following:

        {code}
        <route>
            <from uri="timer:foo?fixedRate=true&amp;period=10000" />
            <inOut uri="activemq:queue:test1" />
            <inOut uri="activemq:queue:test2" />
        </route>
        {code}

        The camel-jms consumer listening on test1 will set the {{JMSCorrelationID}} header to the {{JMSMessageID}} for that exchange.

        When the response returns to the route and sent to test2, this consumer also uses the *same* {{JMSCorrelationID}} as before.

        IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

        Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:
        \\
        {code}
        <route>
                    <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                    <setBody><constant>1,2,3,4,5</constant></setBody>
                    <inOut uri="activemq:queue:test1" />
                    <split parallelProcessing="false">
                        <tokenize token="," />
                        <inOut uri="activemq:queue:test2" />
                    </split>
                    <to uri="log:Finished?showAll=true" />
                </route>
        {code}

        There are several solutions here:

        # Remove the JMSCorrelationID header before each request to test2
        # Enable the useMessageIDAsCorrelationID option to true on both producer and consumer
        # Create a new option to assign a unique ID to the JMSCorrelationID header

        The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me this is inefficient and flaky for enterprise deployments.

        That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

        I propose calling this option 'assignUniqueCorrelationID'.

        I'm happy to submit a patch for this, targeting 2.10.1.
        Imagine a sequential JMS invocation across Camel routes, such as the following:

        {code}
        <route>
            <from uri="timer:foo?fixedRate=true&amp;period=10000" />
            <inOut uri="activemq:queue:test1" />
            <inOut uri="activemq:queue:test2" />
        </route>
        {code}

        The camel-jms consumer listening on test1 will set the {{JMSCorrelationID}} header to the {{JMSMessageID}} for that exchange.

        When the response returns to the route and sent to test2, this consumer also uses the *same* {{JMSCorrelationID}} as before.

        IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

        Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:
        \\
        {code}
                <route>
                    <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                    <setBody><constant>1,2,3,4,5</constant></setBody>
                    <inOut uri="activemq:queue:test1" />
                    <split parallelProcessing="true">
                        <tokenize token="," />
                        <inOut uri="activemq:queue:test2" />
                    </split>
                    <to uri="log:Finished?showAll=true" />
                </route>
                
                <route>
                    <from uri="activemq:queue:test1" />
                    <to uri="log:Received?showAll=true" />
                </route>
         
                 <route>
                    <from uri="activemq:queue:test2" />
                    <to uri="log:Received?showAll=true" />
                    <setBody><constant>reply</constant></setBody>
                    <delay><constant>100</constant></delay>
                </route>
        {code}

        There are several solutions here:

        # Remove the JMSCorrelationID header before each request to test2
        # Enable the useMessageIDAsCorrelationID option to true on both producer and consumer
        # Create a new option to assign a unique ID to the JMSCorrelationID header

        The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me this is inefficient and flaky for enterprise deployments.

        That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

        I propose calling this option 'assignUniqueCorrelationID'.

        I'm happy to submit a patch for this, targeting 2.10.1.
        Raúl Kripalani made changes -
        Field Original Value New Value
        Description Imagine a sequential JMS invocation across Camel routes, such as the following:

        {code}
        <route>
            <from uri="timer:foo?fixedRate=true&amp;period=10000" />
            <inOut uri="activemq:queue:test1" />
            <inOut uri="activemq:queue:test2" />
        </route>
        {code}

        The camel-jms consumer listening on test1 will set the JMSCorrelationID header to the JMSMessageID for that exchange.

        When the response returns to the route and sent to test2, this consumer also uses the *same* JMSCorrelationID as before.

        IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

        Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:

        {code}
        <route>
                    <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                    <setBody><constant>1,2,3,4,5</constant></setBody>
                    <inOut uri="activemq:queue:test1" />
                    <split parallelProcessing="false">
                        <tokenize token="," />
                        <inOut uri="activemq:queue:test2" />
                    </split>
                    <to uri="log:Finished?showAll=true" />
                </route>
        {code}

        There are several solutions here:

        #. Remove the JMSCorrelationID header before each request to test2
        #. Enable the useMessageIDAsCorrelationID option to true on both producer and consumer
        #. Create a new option to assign a unique ID to the JMSCorrelationID header

        The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me it is not efficient at all.

        That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

        I propose calling this option 'assignUniqueCorrelationID'.

        I'm happy to submit a patch for this, targeting 2.10.1.
        Imagine a sequential JMS invocation across Camel routes, such as the following:

        {code}
        <route>
            <from uri="timer:foo?fixedRate=true&amp;period=10000" />
            <inOut uri="activemq:queue:test1" />
            <inOut uri="activemq:queue:test2" />
        </route>
        {code}

        The camel-jms consumer listening on test1 will set the {{JMSCorrelationID}} header to the {{JMSMessageID}} for that exchange.

        When the response returns to the route and sent to test2, this consumer also uses the *same* {{JMSCorrelationID}} as before.

        IMHO, this behaviour is incorrect from the pragmatic perspective. Why should test2 reuse the Message ID from test1 after all?

        Despite that, the correlation works for simple cases, but for complex routing with parallel JMS exchanges, it doesn't because the correlation ID is not unique for each exchange. Consider the following splitter:
        \\
        {code}
        <route>
                    <from uri="timer:foo?fixedRate=true&amp;period=10000" />
                    <setBody><constant>1,2,3,4,5</constant></setBody>
                    <inOut uri="activemq:queue:test1" />
                    <split parallelProcessing="false">
                        <tokenize token="," />
                        <inOut uri="activemq:queue:test2" />
                    </split>
                    <to uri="log:Finished?showAll=true" />
                </route>
        {code}

        There are several solutions here:

        # Remove the JMSCorrelationID header before each request to test2
        # Enable the useMessageIDAsCorrelationID option to true on both producer and consumer
        # Create a new option to assign a unique ID to the JMSCorrelationID header

        The downside of 2 is that the message ID is only assigned after the JMS dispatch, so the component updates the correlation map with the final JMSMessageID after the JMS dispatch. Also, camel-jms is also prepared to handle cases where the reply comes in before the map update has occurred, by waiting 5 seconds if an unknown correlation ID is received, blocking the receipt of any further replies. You will agree with me this is inefficient and flaky for enterprise deployments.

        That's where the upside of 3 lies. By expressly setting a static unique ID as the correlation ID, we get rid of these race conditions and inefficiencies.

        I propose calling this option 'assignUniqueCorrelationID'.

        I'm happy to submit a patch for this, targeting 2.10.1.
        Raúl Kripalani created issue -

          People

          • Assignee:
            Raúl Kripalani
            Reporter:
            Raúl Kripalani
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development