Camel
  1. Camel
  2. CAMEL-4770

Add startAsync option to JMS consumer endpoint to allow route to be started, but the connection to the remote broker occurs async

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.10.0
    • Component/s: camel-jms
    • Labels:
      None
    • Estimated Complexity:
      Advanced

      Description

      This allows people to add routes which consumes from JMS destinations, by which the consumer should start async in a separate thread, this ensures Camel will continue starting the route, and continue the code. Then the asyns thread starts the JmsConsumer in the background. This is needed in case the remote connection to the JMS broker does not work. But you want to signal the route is started anyway, as the JMS consumer most likely support failover and retry, so the connection may come online later.

      We may need to add some way to store a flag, so you from JMX can monitor if the JMS consumer is live or not.

      See nabble
      http://camel.465427.n5.nabble.com/Adding-JMS-route-may-block-if-remote-is-down-using-failover-connection-tp5037014p5037014.html

      1. JMXFlag.patch
        5 kB
        Michał Warecki
      2. AsyncStartListener2.patch
        4 kB
        Michał Warecki

        Activity

        Hide
        Michał Warecki added a comment -

        I attach first part of patch. Option "asyncStartListener" (ie. activemq:test?asyncStartListener=true) is responsible for asynchronous start listener in a new thread. I will now try to look at how we can connect this option to JMX.

        Show
        Michał Warecki added a comment - I attach first part of patch. Option "asyncStartListener" (ie. activemq:test?asyncStartListener=true) is responsible for asynchronous start listener in a new thread. I will now try to look at how we can connect this option to JMX.
        Hide
        Claus Ibsen added a comment -

        Thanks for your work on this.

        Can you use a non scheduled thread pool, as a scheduled thread pool cannot shrink, so it just keep 1 thread alive all the time, which is really not needed. We essentially need a thread pool that has 0-1 in size, so the thread can terminate when no longer needed. Also as a good citizen you ought to shutdown the pool when the consumer is shutting down in its doShutdown method. Albeit Camel has a fallback mehanishm, so it will shutdown thread pools when it shutdown itself. But still being a good citizen is a good idea.

        Also if you use maven from the command line, you may want to check about building with checkstyle, so the source code is formatted correct.
        http://camel.apache.org/building.html

        Show
        Claus Ibsen added a comment - Thanks for your work on this. Can you use a non scheduled thread pool, as a scheduled thread pool cannot shrink, so it just keep 1 thread alive all the time, which is really not needed. We essentially need a thread pool that has 0-1 in size, so the thread can terminate when no longer needed. Also as a good citizen you ought to shutdown the pool when the consumer is shutting down in its doShutdown method. Albeit Camel has a fallback mehanishm, so it will shutdown thread pools when it shutdown itself. But still being a good citizen is a good idea. Also if you use maven from the command line, you may want to check about building with checkstyle, so the source code is formatted correct. http://camel.apache.org/building.html
        Hide
        Michał Warecki added a comment -

        Thanks for the tips. I'll work on it yet. Also I'll activate checkstyle in eclipse.

        Show
        Michał Warecki added a comment - Thanks for the tips. I'll work on it yet. Also I'll activate checkstyle in eclipse.
        Hide
        Michał Warecki added a comment -

        I attach new version of the patch. I hope I interpreted your instructions well. If this is correct, I will begin work on a connection with JMX.

        Show
        Michał Warecki added a comment - I attach new version of the patch. I hope I interpreted your instructions well. If this is correct, I will begin work on a connection with JMX.
        Hide
        Claus Ibsen added a comment -

        Thanks Michael.

        I decided to use a shared cached thread pool on JmsComponent, this is to ensure each startup task gets a dedicated thread to run, as they can potentially take a long time and/or block. Also the cached thread pool will shrink when the task is no longer in use. And its better to have a shared thread pool, than a thread pool per consumer, that could add a lot of thread pools for people with many JMS routes.

        Show
        Claus Ibsen added a comment - Thanks Michael. I decided to use a shared cached thread pool on JmsComponent, this is to ensure each startup task gets a dedicated thread to run, as they can potentially take a long time and/or block. Also the cached thread pool will shrink when the task is no longer in use. And its better to have a shared thread pool, than a thread pool per consumer, that could add a lot of thread pools for people with many JMS routes.
        Hide
        Michał Warecki added a comment -

        Great solution. At the start of Camel excess threads could cause problems with memory. I think I will take into account the problem of large number of threads in the pool while designing an alerts mechanism (http://camel.465427.n5.nabble.com/DISCUSS-Camel-Alerts-td5497221.html).

        I think we should open a new ticket for the new JMX flags because, due to unified management solution in the Camel, it will cover all consumers. One solution would be to add new interface (extending org.apache.camel.Service) with method isRunning() or isAlive() and add the same method to org.apache.camel.management.mbean.MenagedService and org.apache.camel.api.management.mbean.ManagedServiceMBean. Implementation would be like in org.apache.camel.CamelContext.ManagedService#getState. I will try to implement this solution when I find some free time.

        Show
        Michał Warecki added a comment - Great solution. At the start of Camel excess threads could cause problems with memory. I think I will take into account the problem of large number of threads in the pool while designing an alerts mechanism ( http://camel.465427.n5.nabble.com/DISCUSS-Camel-Alerts-td5497221.html ). I think we should open a new ticket for the new JMX flags because, due to unified management solution in the Camel, it will cover all consumers. One solution would be to add new interface (extending org.apache.camel.Service) with method isRunning() or isAlive() and add the same method to org.apache.camel.management.mbean.MenagedService and org.apache.camel.api.management.mbean.ManagedServiceMBean. Implementation would be like in org.apache.camel.CamelContext.ManagedService#getState. I will try to implement this solution when I find some free time.
        Hide
        Michał Warecki added a comment -

        I prepared a draft implementation of JMX flag specifying whether consumer is actively running. I'm still confused about the naming used so if you have better proposition feel free
        Empirically tested so if the implementation is correct I can prepare unit tests.

        Show
        Michał Warecki added a comment - I prepared a draft implementation of JMX flag specifying whether consumer is actively running. I'm still confused about the naming used so if you have better proposition feel free Empirically tested so if the implementation is correct I can prepare unit tests.
        Hide
        Claus Ibsen added a comment -

        I am not keen on adding more APIs to camel-core, just for this little need here.

        So for now lets leave it as is. If there is any JMX stats needed, then it should be JmsConsumer specific only.

        Show
        Claus Ibsen added a comment - I am not keen on adding more APIs to camel-core, just for this little need here. So for now lets leave it as is. If there is any JMX stats needed, then it should be JmsConsumer specific only.

          People

          • Assignee:
            Claus Ibsen
            Reporter:
            Claus Ibsen
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development