Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-10476

configAdminFile not used to populate property placeholders in camel-test-blueprint when run via camel-maven-plugin

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.15.3, 2.16.0, 2.15.4, 2.16.1, 2.15.5, 2.15.6, 2.16.2, 2.16.3, 2.16.4, 2.17.0, 2.17.1, 2.17.2, 2.17.3, 2.18.0
    • Fix Version/s: 2.16.5, 2.17.4, 2.18.1, 2.19.0
    • Component/s: camel-blueprint
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      Problem: When running with a Camel Blueprint project a configAdminFile is not used to populate propertyplacehoders in camel-test-blueprint when exectued with camel-maven-plugin(camel:run). So a user can't run camel locally in a similar way to running in Karaf with file based property placeholder values.

      Workaround: I think, but haven't tested yet, that you can work around this locally using the methods described here: http://ggrzybek.blogspot.com/2015/12/camel-blueprint-test-support.html and/or how this solution https://github.com/cschneider/Karaf-Tutorial/tree/master/camel/order/src appears to use exec:java locally and loads the properties via PropertiesComponent.

      To reproduce the problem:
      Create a new project using camel-archetype-blueprint. (You need to change the log4j config to make it run.) To reduce the time, I created a project that runs here: https://github.com/ryanco/propertyconfig. Instead of using a default in the blueprint XML for the propertyplaceholder, I setup the POM to include the following:

            <plugin>
              <groupId>org.apache.camel</groupId>
              <artifactId>camel-maven-plugin</artifactId>
              <version>2.18.0</version>
              <configuration>
      	      <useBlueprint>true</useBlueprint
               <configAdminPid>com.yarsquidy.props.propertyconfig</configAdminPid>
                <configAdminFileName>etc/com.yarsquidy.props.propertyconfig</configAdminFileName>
              </configuration>
            </plugin>
      

      In Camel 2.15.2 or earlier, this file would be loaded when mvn camel:run was invoked and the properties would be available via the PID at run time. After the changes made in CAMEL-9313, it appears that the method org.apache.camel.test.blueprint.CamelBlueprintHelper#setPersistentFileForConfigAdmin is only called in when the createTestBundle pathway is taken in org.apache.camel.test.blueprint.CamelBlueprintHelper#createBundleContext(java.lang.String, java.lang.String, boolean, java.lang.String, java.lang.String, java.lang.String, java.lang.String[]...). So it appears test using CamelBlueprintTestSupport get this functionality (as shown by the tests) but things executed from camel:run do not.

      Here you can see in Camel 2.14 that call to org.apache.camel.test.blueprint.CamelBlueprintHelper#setPersistentFileForConfigAdmin is made after the bundelContext is created.
      https://github.com/apache/camel/blob/camel-2.14.x/components/camel-test-blueprint/src/main/java/org/apache/camel/test/blueprint/Main.java#L103

      In the master branch version, that call is no longer made from main after the context is returned.
      https://github.com/apache/camel/blob/master/components/camel-test-blueprint/src/main/java/org/apache/camel/test/blueprint/Main.java#L106

      I made a change locally to add a similar call to org.apache.camel.test.blueprint.CamelBlueprintHelper#setPersistentFileForConfigAdmin in Camel 2.18:

      LOG.debug("Starting Blueprint XML file: " + descriptors);
      if (configAdminPid != null && configAdminFileName != null) {
      		// pid/file is used to set INITIAL content of ConfigAdmin to be used when blueprint container is started
      		LOG.info("ConfigAdminPid and ConfigAdminFileName are not null");
      		bundleContext = createBundleContext(bundleName, new String[] {configAdminFileName, configAdminPid});
      } else {
      		bundleContext = createBundleContext(bundleName);
      }
      CamelBlueprintHelper.setPersistentFileForConfigAdmin(bundleContext, configAdminPid, configAdminFileName, new Properties(), null, null, false);
      

      Here is the output of the log statement from the example before this change:

      [ntext          INFO  Apache Camel 2.18.0 (CamelContext: blueprint-bean-context) started in 0.214 seconds
      [ntext) thread #0 - timer://foo] timerToLog                     INFO  The message contains ${greeting} at 2016-11-14 08:42:03
      [ntext) thread #0 - timer://foo] timerToLog                     INFO  The message contains ${greeting} at 2016-11-14 08:42:08
      

      Here is the output of the log statement from the example after this change:

      [         Blueprint Extender: 3] BlueprintCamelContext          INFO  Apache Camel 2.18.1-SNAPSHOT (CamelContext: blueprint-bean-context) started in 0.257 seconds
      [ntext) thread #0 - timer://foo] timerToLog                     INFO  The message contains Hello From File! at 2016-11-14 08:54:09
      [ntext) thread #0 - timer://foo] timerToLog                     INFO  The message contains Hello From File! at 2016-11-14 08:54:14
      

      As you can see before the change, the $

      {greeting}

      property is not poplulated via propertyplacehoder. After the change it is replaced.

      Given all the discussion of timing related issues in CAMEL-9313, I'm hesitant to say this is a good enough solution or that it aligns with the intention of the changes made in that fix. Given that configAdminFileName and configAdminPid are passed into createBundleContext, perhaps the call to org.apache.camel.test.blueprint.CamelBlueprintHelper#setPersistentFileForConfigAdmin should happen inside createBundleContext or one of it sub-methods.

      Overall, I "think" a user should be able to use the configAdminPid and configAdminFileName settings to load properties via camel:run rather than work aound it, but I could be persumptious there.

        Issue Links

          Activity

          Hide
          gzres Grzegorz Grzybek added a comment -

          Fixed in:

          There were already org.apache.camel.test.blueprint.MainTest, org.apache.camel.test.blueprint.MainNoPidTest and org.apache.camel.test.blueprint.MainNoReloadTest, but all of them explicitly used:

          main.setIncludeSelfAsBundle(true);
          

          Now I've added org.apache.camel.test.blueprint.MainTest#testMainWithoutIncludingTestBundle() which calls main.setIncludeSelfAsBundle(false).

          Show
          gzres Grzegorz Grzybek added a comment - Fixed in: master camel-2.18.x camel-2.17.x camel-2.16.x There were already org.apache.camel.test.blueprint.MainTest , org.apache.camel.test.blueprint.MainNoPidTest and org.apache.camel.test.blueprint.MainNoReloadTest , but all of them explicitly used: main.setIncludeSelfAsBundle( true ); Now I've added org.apache.camel.test.blueprint.MainTest#testMainWithoutIncludingTestBundle() which calls main.setIncludeSelfAsBundle(false) .
          Hide
          gzres Grzegorz Grzybek added a comment - - edited

          Ah, one more thing. org.apache.camel.test.blueprint.CamelBlueprintHelper#createTestBundle() creates TinyBundle without Bundle-Blueprint header. Thus org.apache.aries.blueprint.container.BlueprintExtender#getBlueprintPaths() explicitly uses:

          Bundle-Blueprint: OSGI-INF/blueprint/
          

          which eventually translates to OSGI-INF/blueprint/*.xml.

          And because mvn camel:run doesn't set includeSelfAsBundle, setting descriptors (<applicationContextUri> or <fileApplicationContextUri>) doesn't make sense with <useBlueprint> == true.

          Show
          gzres Grzegorz Grzybek added a comment - - edited Ah, one more thing. org.apache.camel.test.blueprint.CamelBlueprintHelper#createTestBundle() creates TinyBundle without Bundle-Blueprint header. Thus org.apache.aries.blueprint.container.BlueprintExtender#getBlueprintPaths() explicitly uses: Bundle-Blueprint: OSGI-INF/blueprint/ which eventually translates to OSGI-INF/blueprint/*.xml . And because mvn camel:run doesn't set includeSelfAsBundle , setting descriptors ( <applicationContextUri> or <fileApplicationContextUri> ) doesn't make sense with <useBlueprint> == true .
          Hide
          gzres Grzegorz Grzybek added a comment -

          Because mvn camel:run is not a test, for such scenarios, includeTestBundle should always equal to false. That's why I'll separate "test bundle" and a "trick bundle" that initializes configadmin before running blueprint.

          Show
          gzres Grzegorz Grzybek added a comment - Because mvn camel:run is not a test, for such scenarios, includeTestBundle should always equal to false . That's why I'll separate "test bundle" and a "trick bundle" that initializes configadmin before running blueprint.
          Hide
          gzres Grzegorz Grzybek added a comment - - edited

          The problem is org.apache.camel.test.blueprint.Main#includeSelfAsBundle field. Or rather the fact that it's not set in org.apache.camel.maven.RunMojo#execute().

          All JUnit tests (extending from org.apache.camel.test.blueprint.CamelBlueprintTestSupport) by default use org.apache.camel.test.blueprint.CamelBlueprintTestSupport#includeTestBundle() which returns true.

          When you simply do mvn clean test, neither target/classes nor target/test-classess contain META-INF/MANIFEST.MF file which is necessary to treat the location as "bundle" to be picked up in org.apache.camel.test.blueprint.CamelBlueprintHelper#getBundleDescriptors():

              public List<BundleDescriptor> scanForBundles(String filterString, ClassLoader loader)
                      throws Exception
              {
          …
                  loader = (loader != null) ? loader : getClass().getClassLoader();
          
          …
                  for (Enumeration<URL> e = loader.getResources(
                          "META-INF/MANIFEST.MF"); e.hasMoreElements(); )
                  {
          …
          

          That's why in JUnit tests, explicit bundle is created using org.apache.camel.test.blueprint.CamelBlueprintHelper#createTestBundle(). Only this method does the "initial configadmin file" trick (see here for details).

          When running using mvn clean camel:run you'll simply get java.lang.RuntimeException: Gave up waiting for service (objectClass=org.apache.camel.CamelContext), because no "bundle" provides your blueprint XML descriptor.
          You need at least mvn clean package camel:run, so you have these "bundles" installed by felix-connect (a bundle with blueprint descriptor is #1):

          0 = {org.apache.felix.connect.launch.BundleDescriptor@5887} "jar:file:/opt/java/tools/apache-maven-3.3.9/boot/plexus-classworlds-2.5.2.jar!/"
          1 = {org.apache.felix.connect.launch.BundleDescriptor@5888} "file:/data/ggrzybek/sources/github.com/_other/propertyconfig/target/classes/"
          2 = {org.apache.felix.connect.launch.BundleDescriptor@5889} "jar:file:/home/ggrzybek/.m2/repository/org/apache/camel/camel-core/2.18.0/camel-core-2.18.0.jar!/"
          3 = {org.apache.felix.connect.launch.BundleDescriptor@5890} "jar:file:/home/ggrzybek/.m2/repository/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.jar!/"
          …
          

          When running under JUnit, you have (a bundle with blueprint descriptor is #0):

          0 = {org.apache.felix.connect.launch.BundleDescriptor@1869} "jar:file:/data/ggrzybek/sources/github.com/_other/propertyconfig/target/test-bundles/blueprintbeanroutetest-1479460649241.jar!/"
          1 = {org.apache.felix.connect.launch.BundleDescriptor@1806} "jar:file:/home/ggrzybek/.m2/repository/org/apache/camel/camel-core/2.18.0/camel-core-2.18.0.jar!/"
          2 = {org.apache.felix.connect.launch.BundleDescriptor@1807} "jar:file:/home/ggrzybek/.m2/repository/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.jar!/"
          3 = {org.apache.felix.connect.launch.BundleDescriptor@1808} "jar:file:/home/ggrzybek/.m2/repository/com/sun/xml/bind/jaxb-core/2.2.11/jaxb-core-2.2.11.jar!/"
          …
          

          If both includeTestBundle was true and we used mvn package, we'd have two bundles with blueprint descriptor.

          Manifest from propertyconfig/target/test-bundles/mybundle-1479461020503.jar (created by org.apache.camel.test.blueprint.CamelBlueprintHelper#createTestBundle():

          Manifest-Version: 2
          SwissboxTinybundlesVersion: pax-swissbox-tinybundles-1.3.2
          Bundle-SymbolicName: MyBundle
          Bundle-Version: 1.0.0
          Built-By: ggrzybek
          Bundle-ManifestVersion: 2
          Bundle-Activator: org.apache.camel.test.blueprint.CamelBlueprintHelper
           $TestBundleActivator
          Created-By: pax-swissbox-tinybundles-1.3.2
          Tool: pax-swissbox-tinybundles-1.3.2
          X-Camel-Blueprint-ConfigAdmin-Init: com.yarsquidy.props.propertyconfig
           =file:/data/ggrzybek/sources/github.com/_other/propertyconfig/etc/com
           .yarsquidy.props.propertyconfig
          

          Manifest from propertyconfig/target/classes/META-INF/MANIFEST.MF (created by maven-bundle-plugin):

          Manifest-Version: 1.0
          Bnd-LastModified: 1479460929527
          Build-Jdk: 1.8.0_112
          Built-By: ggrzybek
          Bundle-Blueprint: OSGI-INF/blueprint/blueprint-bean.xml
          Bundle-ManifestVersion: 2
          Bundle-Name: A Camel Blueprint Route
          Bundle-SymbolicName: com.yarsquidy.props.property-config
          Bundle-Version: 1.0.0.SNAPSHOT
          Created-By: Apache Maven Bundle Plugin
          Export-Package: com.yarsquidy.props;version="1.0.0"
          Import-Package: org.osgi.service.blueprint;version="[1.0.0,2.0.0)"
          Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
          Tool: Bnd-3.2.0.201605172007
          

          This is a reason of some problems we had when running JUnit tests after mvn ... package.

          So the solution should be to perform the "initial configadmin file" trick when running with mvn camel:run and without includeTestBundle. camel:run should pick up a bundle containing blueprint descriptor which was created by mvn package

          Also we should prevent using two bundles containing the same descriptors.

          I already have an idea.

          Show
          gzres Grzegorz Grzybek added a comment - - edited The problem is org.apache.camel.test.blueprint.Main#includeSelfAsBundle field. Or rather the fact that it's not set in org.apache.camel.maven.RunMojo#execute() . All JUnit tests (extending from org.apache.camel.test.blueprint.CamelBlueprintTestSupport ) by default use org.apache.camel.test.blueprint.CamelBlueprintTestSupport#includeTestBundle() which returns true . When you simply do mvn clean test , neither target/classes nor target/test-classess contain META-INF/MANIFEST.MF file which is necessary to treat the location as "bundle" to be picked up in org.apache.camel.test.blueprint.CamelBlueprintHelper#getBundleDescriptors() : public List<BundleDescriptor> scanForBundles( String filterString, ClassLoader loader) throws Exception { … loader = (loader != null ) ? loader : getClass().getClassLoader(); … for (Enumeration<URL> e = loader.getResources( "META-INF/MANIFEST.MF" ); e.hasMoreElements(); ) { … That's why in JUnit tests, explicit bundle is created using org.apache.camel.test.blueprint.CamelBlueprintHelper#createTestBundle() . Only this method does the "initial configadmin file" trick (see here for details). When running using mvn clean camel:run you'll simply get java.lang.RuntimeException: Gave up waiting for service (objectClass=org.apache.camel.CamelContext) , because no "bundle" provides your blueprint XML descriptor. You need at least mvn clean package camel:run , so you have these "bundles" installed by felix-connect (a bundle with blueprint descriptor is #1): 0 = {org.apache.felix.connect.launch.BundleDescriptor@5887} "jar:file:/opt/java/tools/apache-maven-3.3.9/boot/plexus-classworlds-2.5.2.jar!/" 1 = {org.apache.felix.connect.launch.BundleDescriptor@5888} "file:/data/ggrzybek/sources/github.com/_other/propertyconfig/target/classes/" 2 = {org.apache.felix.connect.launch.BundleDescriptor@5889} "jar:file:/home/ggrzybek/.m2/repository/org/apache/camel/camel-core/2.18.0/camel-core-2.18.0.jar!/" 3 = {org.apache.felix.connect.launch.BundleDescriptor@5890} "jar:file:/home/ggrzybek/.m2/repository/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.jar!/" … When running under JUnit, you have (a bundle with blueprint descriptor is #0): 0 = {org.apache.felix.connect.launch.BundleDescriptor@1869} "jar:file:/data/ggrzybek/sources/github.com/_other/propertyconfig/target/test-bundles/blueprintbeanroutetest-1479460649241.jar!/" 1 = {org.apache.felix.connect.launch.BundleDescriptor@1806} "jar:file:/home/ggrzybek/.m2/repository/org/apache/camel/camel-core/2.18.0/camel-core-2.18.0.jar!/" 2 = {org.apache.felix.connect.launch.BundleDescriptor@1807} "jar:file:/home/ggrzybek/.m2/repository/org/slf4j/slf4j-api/1.7.21/slf4j-api-1.7.21.jar!/" 3 = {org.apache.felix.connect.launch.BundleDescriptor@1808} "jar:file:/home/ggrzybek/.m2/repository/com/sun/xml/bind/jaxb-core/2.2.11/jaxb-core-2.2.11.jar!/" … If both includeTestBundle was true and we used mvn package , we'd have two bundles with blueprint descriptor. Manifest from propertyconfig/target/test-bundles/mybundle-1479461020503.jar (created by org.apache.camel.test.blueprint.CamelBlueprintHelper#createTestBundle() : Manifest-Version: 2 SwissboxTinybundlesVersion: pax-swissbox-tinybundles-1.3.2 Bundle-SymbolicName: MyBundle Bundle-Version: 1.0.0 Built-By: ggrzybek Bundle-ManifestVersion: 2 Bundle-Activator: org.apache.camel.test.blueprint.CamelBlueprintHelper $TestBundleActivator Created-By: pax-swissbox-tinybundles-1.3.2 Tool: pax-swissbox-tinybundles-1.3.2 X-Camel-Blueprint-ConfigAdmin-Init: com.yarsquidy.props.propertyconfig =file:/data/ggrzybek/sources/github.com/_other/propertyconfig/etc/com .yarsquidy.props.propertyconfig Manifest from propertyconfig/target/classes/META-INF/MANIFEST.MF (created by maven-bundle-plugin): Manifest-Version: 1.0 Bnd-LastModified: 1479460929527 Build-Jdk: 1.8.0_112 Built-By: ggrzybek Bundle-Blueprint: OSGI-INF/blueprint/blueprint-bean.xml Bundle-ManifestVersion: 2 Bundle-Name: A Camel Blueprint Route Bundle-SymbolicName: com.yarsquidy.props.property-config Bundle-Version: 1.0.0.SNAPSHOT Created-By: Apache Maven Bundle Plugin Export-Package: com.yarsquidy.props;version="1.0.0" Import-Package: org.osgi.service.blueprint;version="[1.0.0,2.0.0)" Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))" Tool: Bnd-3.2.0.201605172007 This is a reason of some problems we had when running JUnit tests after mvn ... package . So the solution should be to perform the "initial configadmin file" trick when running with mvn camel:run and without includeTestBundle . camel:run should pick up a bundle containing blueprint descriptor which was created by mvn package Also we should prevent using two bundles containing the same descriptors. I already have an idea.
          Hide
          ryanco Ryan Colwell added a comment -

          Let me know if I can provide any more information or if something isn't clear. I also was able to successfully test the workaround and I can run locally(in an IDE) using java:exec with the same properties file that the unit tests load by overriding loadConfigAdminConfigurationFile and reading that config file. The only thing I am not confident in with the workaround is the difference between running in a DefaultCamelContext vs a BlueprintCamelContext in the IDE.

          Show
          ryanco Ryan Colwell added a comment - Let me know if I can provide any more information or if something isn't clear. I also was able to successfully test the workaround and I can run locally(in an IDE) using java:exec with the same properties file that the unit tests load by overriding loadConfigAdminConfigurationFile and reading that config file. The only thing I am not confident in with the workaround is the difference between running in a DefaultCamelContext vs a BlueprintCamelContext in the IDE.
          Hide
          gzres Grzegorz Grzybek added a comment -

          Thanks for detailed description. I'll have a look at the end of this week.

          Show
          gzres Grzegorz Grzybek added a comment - Thanks for detailed description. I'll have a look at the end of this week.

            People

            • Assignee:
              gzres Grzegorz Grzybek
              Reporter:
              ryanco Ryan Colwell
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development