Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-9313

CamelBlueprintTestSupport - can't initialize ConfigAdmin configurations

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.15.4
    • Fix Version/s: 2.15.6, 2.16.2, 2.17.0
    • Component/s: camel-blueprint
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      The bugfix CAMEL-8948 seems to make older timing problems related to property-placeholders visible.

      To reproduce the problem i changed the test
      org.apache.camel.test.blueprint.ConfigAdminNoReloadLoadConfigurationFileTest
      from the component camel-test-blueprint a bit, respectively the context
      "org/apache/camel/test/blueprint/configadmin-no-reload-loadfile.xml":

      I added the trace Attribute to the camelContext:

      <camelContext xmlns="http://camel.apache.org/schema/blueprint" trace="{{tracing}}"> 
      

      and added also the property to the etc/stuff.cfg

      tracing=true 
      

      Until 2.15.2 this worked fine. From 2.15.3 on the property cannot be
      replaced any more.
      But, if setting a breakpoint in CamelBlueprintTestSupport#createBundleContext at loadConfigAdminConfigurationFile() (Line 123 in 2.15.4) - the error occurs even in older versions like 2.14 - so the timing problem seems to be there for a while but did not occur because the loading of the configAdminFile seems to be faster than the event handling during service registration triggered by the code some lines above.

      The issue can also be reproduced when replacing a property's String type
      with int in the MyCoolBean class and setting its value by using the
      placeholder like before but with an int value of course. The test run shows
      that the placeholder "${..}" will not be replaced and leads to a
      NumberfFormatException.

      The production code that is under test works fine in karaf.

      1. camel-9313.patch
        2 kB
        Andreas Siepert

        Issue Links

          Activity

          Hide
          gzres Grzegorz Grzybek added a comment - - edited

          Thanks for the analysis Andreas Siepert.

          You're right - this has changed after 2.15.2. And it's true that if you put a breakpoint (to hold "main" thread) in createBundleContext(), it fails in 2.15.2 too.

          The reason is this line.

          The changes I've made are result of very unpredictable results of camel-test-blueprint tests in CI.

          To describe the problem, be aware that there are 3 groups of threads involved:

          1. main thread, where test is run - this is started by JUnit, maven-surefire-plugin, etc.
          2. threads related to aries blueprint
          3. threads related to felix configadmin (sometimes even started with new Thread(...).start().

          My ultimate goal was to achieve (at least) 100% reliability of the test - to eliminate all race conditions. And the only way to do it is to do some kind of synchronization - using java.util.concurrent primitives and listeners to BP / configadmin events.

          So, when you add trace="{{tracing} }" to <camelContext>, then in 2.15.2 there's great chance (unless you put a breakpoint to hold main thread), that BP container will get initialized after this:

          if (file != null) {
            if (!new File(file[0]).exists()) {
              throw new IllegalArgumentException("The provided file \"" + file[0] + "\" from loadConfigAdminConfigurationFile doesn't exist");
            }
            CamelBlueprintHelper.setPersistentFileForConfigAdmin(answer, file[1], file[0], props, symbolicName, bpEvents, expectReload);
          }
          

          This leads to invocation of org.osgi.service.cm.Configuration#update(java.util.Dictionary<java.lang.String, ?>) and when (and there's high probability of this in 2.15.2) BP container isn't yet started, it'll have those properties already available when <cm:property-placeholder> is initialized:

          public void init() throws Exception {
              ...
              synchronized (lock) {
                  managedObjectManager.register(this, props);
                  Configuration config = CmUtils.getConfiguration(configAdmin, persistentId);
                  if (config != null) {
                      properties = config.getProperties();
                  }
                  updated(properties);
              }
          }
          

          Plus remember - there are two kinds of property placeholders: from Camel and from Aries-Blueprint. Camel version delegates (in OSGi) to Aries version.

          In order to have working trace="{{tracing} }" in 2.15.3+, you have to override simple method: org.apache.camel.test.junit4.CamelTestSupport#useOverridePropertiesWithPropertiesComponent.

          Add:

              @Override
              protected Properties useOverridePropertiesWithPropertiesComponent() {
                  Properties props = new Properties();
                  props.setProperty("tracing", "true");
                  return props;
              }
          

          to org.apache.camel.test.blueprint.ConfigAdminNoReloadLoadConfigurationFileTest and trace="{{tracing} }" to org/apache/camel/test/blueprint/configadmin-no-reload-loadfile.xml and everything will work, because PropertyComponent will have needed property when CamelContext is created.

          In other words - my change in 2.15.3 related to predictability of camel-test-blueprint tests can be summarized like this:

          • we don't test (because of explicit synchronization) the scenarios, when configadmin configurations are updated before blueprint container starts
          • we test scenarios, where configadmin configuration updates should (when there's <cm:property-placeholder ... update-strategy="reload">) lead to reinitialization of blueprint container - and to make it predictable, we have to synchrozize BP and configadmin events

          It worked in 2.15.2 as a kind of side effect - just like the configadmin configuration already existed when blueprint container was created for the first time. but it's not different then specifying it explicitly with:

            <cm:property-placeholder persistent-id="stuff" update-strategy="none">
              <cm:default-properties>
                <cm:property name="x" value="y"/>
              </cm:default-properties>
            </cm:property-placeholder>
          

          And yet another words: CamelBlueprintTestSupport calls org.osgi.service.cm.Configuration#update(java.util.Dictionary<java.lang.String,?>) only to change configadmin configs, not to init them.

          I hope this resolves your issue.

          Show
          gzres Grzegorz Grzybek added a comment - - edited Thanks for the analysis Andreas Siepert . You're right - this has changed after 2.15.2. And it's true that if you put a breakpoint (to hold "main" thread) in createBundleContext() , it fails in 2.15.2 too. The reason is this line . The changes I've made are result of very unpredictable results of camel-test-blueprint tests in CI. To describe the problem, be aware that there are 3 groups of threads involved: main thread, where test is run - this is started by JUnit, maven-surefire-plugin, etc. threads related to aries blueprint threads related to felix configadmin (sometimes even started with new Thread(...).start() . My ultimate goal was to achieve (at least) 100% reliability of the test - to eliminate all race conditions. And the only way to do it is to do some kind of synchronization - using java.util.concurrent primitives and listeners to BP / configadmin events. So, when you add trace="{{tracing} }" to <camelContext> , then in 2.15.2 there's great chance (unless you put a breakpoint to hold main thread), that BP container will get initialized after this : if (file != null ) { if (! new File(file[0]).exists()) { throw new IllegalArgumentException( "The provided file \" " + file[0] + " \ " from loadConfigAdminConfigurationFile doesn't exist" ); } CamelBlueprintHelper.setPersistentFileForConfigAdmin(answer, file[1], file[0], props, symbolicName, bpEvents, expectReload); } This leads to invocation of org.osgi.service.cm.Configuration#update(java.util.Dictionary<java.lang.String, ?>) and when (and there's high probability of this in 2.15.2) BP container isn't yet started, it'll have those properties already available when <cm:property-placeholder> is initialized : public void init() throws Exception { ... synchronized (lock) { managedObjectManager.register( this , props); Configuration config = CmUtils.getConfiguration(configAdmin, persistentId); if (config != null ) { properties = config.getProperties(); } updated(properties); } } Plus remember - there are two kinds of property placeholders: from Camel and from Aries-Blueprint. Camel version delegates (in OSGi) to Aries version. In order to have working trace="{{tracing} }" in 2.15.3+, you have to override simple method: org.apache.camel.test.junit4.CamelTestSupport#useOverridePropertiesWithPropertiesComponent . Add: @Override protected Properties useOverridePropertiesWithPropertiesComponent() { Properties props = new Properties(); props.setProperty( "tracing" , " true " ); return props; } to org.apache.camel.test.blueprint.ConfigAdminNoReloadLoadConfigurationFileTest and trace="{{tracing} }" to org/apache/camel/test/blueprint/configadmin-no-reload-loadfile.xml and everything will work, because PropertyComponent will have needed property when CamelContext is created. In other words - my change in 2.15.3 related to predictability of camel-test-blueprint tests can be summarized like this: we don't test (because of explicit synchronization) the scenarios, when configadmin configurations are updated before blueprint container starts we test scenarios, where configadmin configuration updates should (when there's <cm:property-placeholder ... update-strategy="reload"> ) lead to reinitialization of blueprint container - and to make it predictable, we have to synchrozize BP and configadmin events It worked in 2.15.2 as a kind of side effect - just like the configadmin configuration already existed when blueprint container was created for the first time. but it's not different then specifying it explicitly with: <cm:property-placeholder persistent-id= "stuff" update-strategy= "none" > <cm:default-properties> <cm:property name= "x" value= "y" /> </cm:default-properties> </cm:property-placeholder> And yet another words: CamelBlueprintTestSupport calls org.osgi.service.cm.Configuration#update(java.util.Dictionary<java.lang.String,?>) only to change configadmin configs, not to init them. I hope this resolves your issue.
          Hide
          gzres Grzegorz Grzybek added a comment -

          Reopening - I have possible fix that allows to initialize configadmin before Blueprint container is initialized.

          Show
          gzres Grzegorz Grzybek added a comment - Reopening - I have possible fix that allows to initialize configadmin before Blueprint container is initialized.
          Hide
          asiepert Andreas Siepert added a comment - - edited

          Hi Grzegorz Grzybek and sorry for the late response of mine. I was just writing the comment as yours came in...

          I mentioned a second problem I attached a patch (camel-9313.patch) based on git branch camel-2.15.5 which addresses the second problem and makes it visible. The problem is that the string of the propertyplaceholder is taken as the value which works fine for strings but not for int for example. Please have a look.

          Stacktrace:

          2015-12-01 08:57:37,415 [int Extender: 1] ERROR BlueprintContainerImpl         - Unable to start blueprint container for bundle ConfigAdminLoadConfigurationFileTest/1.0.0
          org.osgi.service.blueprint.container.ComponentDefinitionException: Error setting property: PropertyDescriptor <name: counter, getter: class org.apache.camel.test.blueprint.MyCoolBean.getCounter(), setter: [class org.apache.camel.test.blueprint.MyCoolBean.setCounter(int)]
          	at org.apache.aries.blueprint.container.BeanRecipe.setProperty(BeanRecipe.java:939)
          	at org.apache.aries.blueprint.container.BeanRecipe.setProperties(BeanRecipe.java:905)
          	at org.apache.aries.blueprint.container.BeanRecipe.setProperties(BeanRecipe.java:886)
          	at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:820)
          	at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:787)
          	at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          	at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88)
          	at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:247)
          	at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:183)
          	at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:682)
          	at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:377)
          	at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:269)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          	at org.apache.aries.blueprint.container.ExecutorServiceWrapper.run(ExecutorServiceWrapper.java:106)
          	at org.apache.aries.blueprint.utils.threading.impl.DiscardableRunnable.run(DiscardableRunnable.java:48)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
          	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          Caused by: java.lang.NumberFormatException: For input string: "${counter}"
          	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
          	at java.lang.Integer.parseInt(Integer.java:481)
          	at java.lang.Integer.valueOf(Integer.java:582)
          	at org.apache.aries.blueprint.container.AggregateConverter.convertFromString(AggregateConverter.java:281)
          	at org.apache.aries.blueprint.container.AggregateConverter.convert(AggregateConverter.java:174)
          	at org.apache.aries.blueprint.container.BlueprintRepository.convert(BlueprintRepository.java:402)
          	at org.apache.aries.blueprint.utils.ReflectionUtils$PropertyDescriptor.convert(ReflectionUtils.java:396)
          	at org.apache.aries.blueprint.utils.ReflectionUtils$MethodPropertyDescriptor.internalSet(ReflectionUtils.java:630)
          	at org.apache.aries.blueprint.utils.ReflectionUtils$PropertyDescriptor.set(ReflectionUtils.java:380)
          	at org.apache.aries.blueprint.container.BeanRecipe.setProperty(BeanRecipe.java:937)
          	... 23 more
          

          With kind regards
          Andreas

          Show
          asiepert Andreas Siepert added a comment - - edited Hi Grzegorz Grzybek and sorry for the late response of mine. I was just writing the comment as yours came in... I mentioned a second problem I attached a patch ( camel-9313.patch ) based on git branch camel-2.15.5 which addresses the second problem and makes it visible. The problem is that the string of the propertyplaceholder is taken as the value which works fine for strings but not for int for example. Please have a look. Stacktrace: 2015-12-01 08:57:37,415 [ int Extender: 1] ERROR BlueprintContainerImpl - Unable to start blueprint container for bundle ConfigAdminLoadConfigurationFileTest/1.0.0 org.osgi.service.blueprint.container.ComponentDefinitionException: Error setting property: PropertyDescriptor <name: counter, getter: class org.apache.camel.test.blueprint.MyCoolBean.getCounter(), setter: [class org.apache.camel.test.blueprint.MyCoolBean.setCounter( int )] at org.apache.aries.blueprint.container.BeanRecipe.setProperty(BeanRecipe.java:939) at org.apache.aries.blueprint.container.BeanRecipe.setProperties(BeanRecipe.java:905) at org.apache.aries.blueprint.container.BeanRecipe.setProperties(BeanRecipe.java:886) at org.apache.aries.blueprint.container.BeanRecipe.internalCreate2(BeanRecipe.java:820) at org.apache.aries.blueprint.container.BeanRecipe.internalCreate(BeanRecipe.java:787) at org.apache.aries.blueprint.di.AbstractRecipe$1.call(AbstractRecipe.java:79) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at org.apache.aries.blueprint.di.AbstractRecipe.create(AbstractRecipe.java:88) at org.apache.aries.blueprint.container.BlueprintRepository.createInstances(BlueprintRepository.java:247) at org.apache.aries.blueprint.container.BlueprintRepository.createAll(BlueprintRepository.java:183) at org.apache.aries.blueprint.container.BlueprintContainerImpl.instantiateEagerComponents(BlueprintContainerImpl.java:682) at org.apache.aries.blueprint.container.BlueprintContainerImpl.doRun(BlueprintContainerImpl.java:377) at org.apache.aries.blueprint.container.BlueprintContainerImpl.run(BlueprintContainerImpl.java:269) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at org.apache.aries.blueprint.container.ExecutorServiceWrapper.run(ExecutorServiceWrapper.java:106) at org.apache.aries.blueprint.utils.threading.impl.DiscardableRunnable.run(DiscardableRunnable.java:48) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang. Thread .run( Thread .java:745) Caused by: java.lang.NumberFormatException: For input string: "${counter}" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang. Integer .parseInt( Integer .java:481) at java.lang. Integer .valueOf( Integer .java:582) at org.apache.aries.blueprint.container.AggregateConverter.convertFromString(AggregateConverter.java:281) at org.apache.aries.blueprint.container.AggregateConverter.convert(AggregateConverter.java:174) at org.apache.aries.blueprint.container.BlueprintRepository.convert(BlueprintRepository.java:402) at org.apache.aries.blueprint.utils.ReflectionUtils$PropertyDescriptor.convert(ReflectionUtils.java:396) at org.apache.aries.blueprint.utils.ReflectionUtils$MethodPropertyDescriptor.internalSet(ReflectionUtils.java:630) at org.apache.aries.blueprint.utils.ReflectionUtils$PropertyDescriptor.set(ReflectionUtils.java:380) at org.apache.aries.blueprint.container.BeanRecipe.setProperty(BeanRecipe.java:937) ... 23 more With kind regards Andreas
          Show
          gzres Grzegorz Grzybek added a comment - Resolved in: master branch (2.17): https://github.com/apache/camel/commit/13760a4a5f1672d54997aa8ede30aefed951505a camel-2.16.x branch: https://github.com/apache/camel/commit/2410bfa5b840fd6dd7de86b32fd3e0dee35061c9 camel-2.15.x branch: https://github.com/apache/camel/commit/bd900b3ce2892ac53830179cce5a1576e1ba2915
          Hide
          gzres Grzegorz Grzybek added a comment -

          Andreas Siepert please see thorough explanation of what was fixed here: http://ggrzybek.blogspot.com/2015/12/camel-blueprint-test-support.html

          Show
          gzres Grzegorz Grzybek added a comment - Andreas Siepert please see thorough explanation of what was fixed here: http://ggrzybek.blogspot.com/2015/12/camel-blueprint-test-support.html

            People

            • Assignee:
              gzres Grzegorz Grzybek
              Reporter:
              asiepert Andreas Siepert
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development