Camel
  1. Camel
  2. CAMEL-3686

Allow to share cache between bundles, and only clear cache when no more bundles access that cache

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0
    • Fix Version/s: Future
    • Component/s: camel-cache
    • Labels:
      None
    • Environment:

      camel-cache 2.4.0-fuse-01-00

      Description

      I am using camel-cache component in serviceMix. Cache endpoint uri is
      "cache://elements?maxElementsInMemory=2&memoryStoreEvictionPolicy=MemoryStoreEvictionPolicy.FIFO&overflowToDisk=false&eternal=false&timeToLiveSeconds=800"

      I have 2 bundles (core.jar, services.jar). Inside those bundles I use

      @EndpointInject(uri = Constants.CACHE_URI)
      ProducerTemplate cacheTemplate;

      cacheTemplate.requestBodyAndHeaders(...)

      core.jar puts and reads elements from cache.
      services.jar only reads elements from cache.

      After deploying both bundles it works fine, but if i uninstall services.jar, cache is "destroyed". core.jar (and all others) can't put objects into cache anymore.

      How could I make all bundles to "share" the same cache?

      1. camel-cache.sharedCacheManagerFactory.patch
        25 kB
        Piotr Klimczak
      2. camel-cache.zip
        48 kB
        Justas Samuolis
      3. diff.txt
        2 kB
        Justas Samuolis

        Activity

        Hide
        Claus Ibsen added a comment -

        This is actually more tricky as you use the cache to share data between bundles. The cache component wasn't designed for that, hence why it clear and stop the cache when the endpoint is being shutdown.

        What we need to do is somehow keep track of how many consumers/producers is accessing the cache. I think that could be difficult to do. And then only stop the cache if the consumers/producers count is zero.

        An alternative to add an option on the endpoint to indicate its a shared cache. Then it's only stopped when you stop the camel-cache component. The caveat is that if you stop both bundles, the data in the cache is kept around, until camel-cache is stopped.

        Show
        Claus Ibsen added a comment - This is actually more tricky as you use the cache to share data between bundles. The cache component wasn't designed for that, hence why it clear and stop the cache when the endpoint is being shutdown. What we need to do is somehow keep track of how many consumers/producers is accessing the cache. I think that could be difficult to do. And then only stop the cache if the consumers/producers count is zero. An alternative to add an option on the endpoint to indicate its a shared cache. Then it's only stopped when you stop the camel-cache component. The caveat is that if you stop both bundles, the data in the cache is kept around, until camel-cache is stopped.
        Hide
        Claus Ibsen added a comment -

        This issue is a bit similar to camel-jetty where we also keep track on number of consumers.

        Show
        Claus Ibsen added a comment - This issue is a bit similar to camel-jetty where we also keep track on number of consumers.
        Hide
        Justas Samuolis added a comment -

        additional parameter "shared" added.

        Show
        Justas Samuolis added a comment - additional parameter "shared" added.
        Hide
        Claus Ibsen added a comment -

        Justas can you try to provide a patch? Eg a txt file which has +/- for lines with the changes. Then its easier to spot what you have done.

        You can google that. And maybe Eclipse has a way as well?

        Show
        Claus Ibsen added a comment - Justas can you try to provide a patch? Eg a txt file which has +/- for lines with the changes. Then its easier to spot what you have done. You can google that. And maybe Eclipse has a way as well?
        Hide
        Piotr Klimczak added a comment -

        Things have change a litle since february.

        Let's start from cache sharing across bundles problem. It will not work as default. This is because CacheManager in DefaultCacheManagerFactory class is created like (since always):

        protected CacheManager createCacheManagerInstance() {
            return CacheManager.create(
                    getClass().getResourceAsStream("/ehcache.xml"));
        }
        

        Then take a look at CacheManager.create method:

        /**
         * The Singleton Instance.
         */
        private static volatile CacheManager singleton;
        
        //some code...
        
        public static CacheManager create(InputStream inputStream) throws CacheException {
            if (singleton != null) {
                return singleton;
            }
            synchronized (CacheManager.class) {
                if (singleton == null) {
                    LOG.debug("Creating new CacheManager with InputStream");
                    singleton = new CacheManager(inputStream);
                }
                return singleton;
            }
        }
        

        As you can see CacheManager is creating its own instance as a singleton stored in static field. This is good for one bundle. But remember that each bundle is running in separate ClassLoader. It means that static fields of one bundle are not visible straight to any other. This is fine for daily usage as cache have to be separated by default.

        Please also note, that CacheManager instance is as default disposed/destroyed during bundle shutdown. So stopping one bundle will close your default CacheManager which is implemented in DefaultCacheManagerFactory of camel-cache component:

        @Override
        protected void doStop() throws Exception {
            // shutdown cache manager when stopping
            if (cacheManager != null) {
                cacheManager.shutdown();
            }
        }
        

        Please note that due to my last changes to camel-cache component, cache is not deleted from CacheManager during route shutdown anymore as it was before (which was extremely bad for people like you and me).

        So in my opinion the best what you can do is to create a separate bundle where you can instantiate the CacheManager and expose it under some getter method of your own interface through OSGi service.

        As you have your own CacheManager singleton accesible through OSGi service reference, you can implement (for each bundle using that cache) your own CacheManagerFactory with constructor which expect to get that referenced CachemManager as a parameter, like this:

        public class YourCacheManagerFactory extends CacheManagerFactory {
            private CacheManager cacheManager;
        
            public YourCacheManagerFactory(CacheManager cacheManager) {
                if (cacheManager == null) {
                    throw new RuntimeCamelException("cacheManager should never be null");
                }
                this.cacheManager = cacheManager;
            }
        
            @Override
            protected synchronized CacheManager createCacheManagerInstance() {
                return cacheManager;
            }
        

        Then it could be referenced to the endpoint like this:

        <camelContext xmlns="http://camel.apache.org/schema/spring">
                <endpoint id="fooCache" uri="cache:foo?cacheManagerFactory=#yourReferencedCacheManagerFactory" />
        </camelContext>
        

        This should solve all your problems.

        Please let me know if it works fine for you. If not, then i will do my best to help you.

        Have a FUN!

        Show
        Piotr Klimczak added a comment - Things have change a litle since february. Let's start from cache sharing across bundles problem. It will not work as default. This is because CacheManager in DefaultCacheManagerFactory class is created like (since always): protected CacheManager createCacheManagerInstance() { return CacheManager.create( getClass().getResourceAsStream( "/ehcache.xml" )); } Then take a look at CacheManager.create method: /** * The Singleton Instance. */ private static volatile CacheManager singleton; //some code... public static CacheManager create(InputStream inputStream) throws CacheException { if (singleton != null ) { return singleton; } synchronized (CacheManager.class) { if (singleton == null ) { LOG.debug( "Creating new CacheManager with InputStream" ); singleton = new CacheManager(inputStream); } return singleton; } } As you can see CacheManager is creating its own instance as a singleton stored in static field. This is good for one bundle. But remember that each bundle is running in separate ClassLoader. It means that static fields of one bundle are not visible straight to any other. This is fine for daily usage as cache have to be separated by default. Please also note, that CacheManager instance is as default disposed/destroyed during bundle shutdown. So stopping one bundle will close your default CacheManager which is implemented in DefaultCacheManagerFactory of camel-cache component: @Override protected void doStop() throws Exception { // shutdown cache manager when stopping if (cacheManager != null ) { cacheManager.shutdown(); } } Please note that due to my last changes to camel-cache component, cache is not deleted from CacheManager during route shutdown anymore as it was before (which was extremely bad for people like you and me). So in my opinion the best what you can do is to create a separate bundle where you can instantiate the CacheManager and expose it under some getter method of your own interface through OSGi service. As you have your own CacheManager singleton accesible through OSGi service reference, you can implement (for each bundle using that cache) your own CacheManagerFactory with constructor which expect to get that referenced CachemManager as a parameter, like this: public class YourCacheManagerFactory extends CacheManagerFactory { private CacheManager cacheManager; public YourCacheManagerFactory(CacheManager cacheManager) { if (cacheManager == null ) { throw new RuntimeCamelException( "cacheManager should never be null " ); } this .cacheManager = cacheManager; } @Override protected synchronized CacheManager createCacheManagerInstance() { return cacheManager; } Then it could be referenced to the endpoint like this: <camelContext xmlns= "http://camel.apache.org/schema/spring" > <endpoint id= "fooCache" uri= "cache:foo?cacheManagerFactory=#yourReferencedCacheManagerFactory" /> </camelContext> This should solve all your problems. Please let me know if it works fine for you. If not, then i will do my best to help you. Have a FUN!
        Hide
        Piotr Klimczak added a comment - - edited

        I have found one problem in my solution: the CacheEndpoint will delete the cache from CacheManager during endpoint/context/bundle shutdown.
        So this have to be fixed to let my solution to work.

        Will fix it soon

        Show
        Piotr Klimczak added a comment - - edited I have found one problem in my solution: the CacheEndpoint will delete the cache from CacheManager during endpoint/context/bundle shutdown. So this have to be fixed to let my solution to work. Will fix it soon
        Hide
        Piotr Klimczak added a comment - - edited

        What was added:
        1. Added shared property to URI config
        2. New unit test to test if CacheManager is still alive after context shutdown when shared == true

        What was fixed:
        1. CacheManager is disposed only when shared == false
        2. Cache is deleted from CacheManager only when shared == false

        Have a FUN!

        Show
        Piotr Klimczak added a comment - - edited What was added: 1. Added shared property to URI config 2. New unit test to test if CacheManager is still alive after context shutdown when shared == true What was fixed: 1. CacheManager is disposed only when shared == false 2. Cache is deleted from CacheManager only when shared == false Have a FUN!
        Hide
        Piotr Klimczak added a comment -

        Recreated the patch as additional assertion was added to unit tests.

        Show
        Piotr Klimczak added a comment - Recreated the patch as additional assertion was added to unit tests.
        Hide
        Claus Ibsen added a comment -

        If the cache manager is shared, then you cannot stop it anymore?
        Because all the camel applications will have: shared=true

        Show
        Claus Ibsen added a comment - If the cache manager is shared, then you cannot stop it anymore? Because all the camel applications will have: shared=true
        Hide
        Piotr Klimczak added a comment - - edited

        Sorry for late answer but i haven't received email notification about your post.
        So if you set your cache to be shared, then the cache manager will not be stopped during bundle shutdown. You have to care care about closing cacheManager by your self. This is because cache manager have to be shared across bundles as an osgi service.

        It is a bit complicated. Let's take a look at simple example.
        Let's say we have 3 bundles:
        a) the one which instantiates cacheManagerFactory and exposes it as an osgi service
        b) the first one, which uses exposed by bundle a) cacheManagerFactory and uses it's cacheManager in it's own cacheManagerFactory (as shown in above example: YourCacheManagerFactory)
        c) the second one, same as b)

        So b) and c) uses shared cacheManager of bundle a) and both are marked as shared. Thanks to this cacheManager is not closed during b) or c) shutdown.
        BUT if you shutdown the a) bundle, then all caches used by b) and c) may gone (dependently on your cacheManagerFactory implementation in bundle a)).

        This is how it suppose to work, but was not tested in full scenario yet. Only partially by some unit tests.
        Please let me know if something is wrong with my idea.
        Better ideas are welcome!

        Show
        Piotr Klimczak added a comment - - edited Sorry for late answer but i haven't received email notification about your post. So if you set your cache to be shared, then the cache manager will not be stopped during bundle shutdown. You have to care care about closing cacheManager by your self. This is because cache manager have to be shared across bundles as an osgi service. It is a bit complicated. Let's take a look at simple example. Let's say we have 3 bundles: a) the one which instantiates cacheManagerFactory and exposes it as an osgi service b) the first one, which uses exposed by bundle a) cacheManagerFactory and uses it's cacheManager in it's own cacheManagerFactory (as shown in above example: YourCacheManagerFactory) c) the second one, same as b) So b) and c) uses shared cacheManager of bundle a) and both are marked as shared. Thanks to this cacheManager is not closed during b) or c) shutdown. BUT if you shutdown the a) bundle, then all caches used by b) and c) may gone (dependently on your cacheManagerFactory implementation in bundle a)). This is how it suppose to work, but was not tested in full scenario yet. Only partially by some unit tests. Please let me know if something is wrong with my idea. Better ideas are welcome!
        Hide
        Piotr Klimczak added a comment -

        Please note, that my solution will not fulfill this requirement: "and only clear cache when no more bundles access that cache" as you have to take care by yourself about closing/clearing cache.
        But I can implement such functionality if you want to.

        Show
        Piotr Klimczak added a comment - Please note, that my solution will not fulfill this requirement: "and only clear cache when no more bundles access that cache" as you have to take care by yourself about closing/clearing cache. But I can implement such functionality if you want to.
        Hide
        Piotr Klimczak added a comment -

        Hi ALL!
        Any feedback here?

        Show
        Piotr Klimczak added a comment - Hi ALL! Any feedback here?
        Hide
        Claus Ibsen added a comment -

        Since the shared cache manager is never stopped, then you would leak resources. For example when you start a bundle again, will it not create a new cache manager, and then you have 2 managers (the old, and the new)

        Show
        Claus Ibsen added a comment - Since the shared cache manager is never stopped, then you would leak resources. For example when you start a bundle again, will it not create a new cache manager, and then you have 2 managers (the old, and the new)
        Hide
        Piotr Klimczak added a comment -

        The CacheManagerFactory have to be set as a shared by this:

        <bean id="cacheManagerFactory" class="org.apache.camel.component.cache.DefaultCacheManagerFactory">
            <property name="shared" value="true" />
        </bean>
        

        This have to be done on the host bundle, that will share the cache.
        Next we have to expose the CacheManagerFactory as an OSGi service.
        The other bundles have to make a reference to that CacheManagerFactory.

        The trick is, that the CacheEndpoint instance will close the CacheManagerFactory only if:
        a) CacheManagerFactory has a property "shared" set to false (default)
        b) CacheManagerFactory has a property "shared" manually set to true and the classloader of CacheEndpoint is same as CacheManagerFactory's classloader.

        In other words, if shared is set to true, only the "owner" of CacheManagerFactory can stop it during bundle shutdown.

        This is how we can avoid closing the CacheManagerFactory by stopping/uninstalling the referencing bundles.

        Please note, that the CacheManagerFactory on the host side will be closed only if it will be used in the host's camel context (to route something).
        If it will be created only by spring and exposed as a service, you have to set the destroy-method (doStop) on bean definition (similar like with datasources).

        Is that ok? Or maybe you still want to track consumers?

        Show
        Piotr Klimczak added a comment - The CacheManagerFactory have to be set as a shared by this: <bean id= "cacheManagerFactory" class= "org.apache.camel.component.cache.DefaultCacheManagerFactory" > <property name= "shared" value= " true " /> </bean> This have to be done on the host bundle, that will share the cache. Next we have to expose the CacheManagerFactory as an OSGi service. The other bundles have to make a reference to that CacheManagerFactory. The trick is, that the CacheEndpoint instance will close the CacheManagerFactory only if: a) CacheManagerFactory has a property "shared" set to false (default) b) CacheManagerFactory has a property "shared" manually set to true and the classloader of CacheEndpoint is same as CacheManagerFactory's classloader. In other words, if shared is set to true, only the "owner" of CacheManagerFactory can stop it during bundle shutdown. This is how we can avoid closing the CacheManagerFactory by stopping/uninstalling the referencing bundles. Please note, that the CacheManagerFactory on the host side will be closed only if it will be used in the host's camel context (to route something). If it will be created only by spring and exposed as a service, you have to set the destroy-method (doStop) on bean definition (similar like with datasources). Is that ok? Or maybe you still want to track consumers?
        Hide
        Piotr Klimczak added a comment -

        For better understanding how it works, please take a look at OSGi test cases inside the patch file.

        Show
        Piotr Klimczak added a comment - For better understanding how it works, please take a look at OSGi test cases inside the patch file.
        Hide
        Claus Ibsen added a comment -

        Piotr can you re-attach your patch and grant [x] in license to ASF. Otherwise we cannot accept or use parts from your patch.

        Show
        Claus Ibsen added a comment - Piotr can you re-attach your patch and grant [x] in license to ASF. Otherwise we cannot accept or use parts from your patch.
        Hide
        Piotr Klimczak added a comment -

        Granted.

        Show
        Piotr Klimczak added a comment - Granted.
        Hide
        Piotr Klimczak added a comment -

        Wondering is there something wrong with my patch (any comment appreciated?) or maybe this issue have a low priority?

        Show
        Piotr Klimczak added a comment - Wondering is there something wrong with my patch (any comment appreciated?) or maybe this issue have a low priority?
        Hide
        Claus Ibsen added a comment -

        The stop logic is not in line what we do in other components.
        The class loader comparison is "special", and we should avoid such things.

        Show
        Claus Ibsen added a comment - The stop logic is not in line what we do in other components. The class loader comparison is "special", and we should avoid such things.

          People

          • Assignee:
            Unassigned
            Reporter:
            Justas Samuolis
          • Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development