Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Tags:
      osgi

      Description

      After adding OSGi metadata to zookeeper.jar it can be used as both an OSGi bundle as well as an ordinary jar file.
      In the CXF/DOSGi project the buildsystem does this using the maven-bundle-plugin: http://svn.apache.org/repos/asf/cxf/dosgi/trunk/discovery/distributed/zookeeper-wrapper/pom.xml

      The MANIFEST.MF generated by maven-bundle-plugin is attached to this bug, this works for the CXF/DOSGi project.
      If your buildsystem isn't using maven, I would advise to use bnd (http://www.aqute.biz/Code/Bnd). BND defines its own ant task in which you should be able to use more or less the same instructions as were used in maven:
      <instructions>
      <Bundle-Name>ZooKeeper bundle</Bundle-Name>
      <Bundle-Description>This bundle contains the ZooKeeper library</Bundle-Description>
      <Bundle-SymbolicName>org.apache.hadoop.zookeeper</Bundle-SymbolicName>
      <Bundle-Version>3.1.1</Bundle-Version>
      <Import-Package>*</Import-Package>
      <Export-Package>*;version=3.1.1</Export-Package>
      </instructions>

      Oh and one other thing. Is it really necessary to put the source code in the Jar file too? I would put that in a separate source distribution

      See also: http://mail-archives.apache.org/mod_mbox/hadoop-zookeeper-user/200905.mbox/%3c4A2009B1.3030603@yahoo-inc.com%3e

      1. buildxmlpatch.patch
        2 kB
        David Bosschaert
      2. zk_patch3.patch
        10 kB
        David Bosschaert
      3. MANIFEST.MF
        4 kB
        David Bosschaert

        Issue Links

          Activity

          Hide
          Patrick Hunt added a comment -

          Chris thanks for the report, new jira created to address: ZOOKEEPER-749

          Show
          Patrick Hunt added a comment - Chris thanks for the report, new jira created to address: ZOOKEEPER-749
          Hide
          Chris Custine added a comment -

          As of 3.3.0, I see the OSGI meta data in the tar.gz download from the Apache mirrors, but it is not in the zookeeper-3.3.0.jar artifact that is deployed to the maven central repository here: http://repo2.maven.org/maven2/org/apache/hadoop/zookeeper/3.3.0/

          Show
          Chris Custine added a comment - As of 3.3.0, I see the OSGI meta data in the tar.gz download from the Apache mirrors, but it is not in the zookeeper-3.3.0.jar artifact that is deployed to the maven central repository here: http://repo2.maven.org/maven2/org/apache/hadoop/zookeeper/3.3.0/
          Hide
          Hudson added a comment -

          Integrated in ZooKeeper-trunk #558 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/558/)

          Show
          Hudson added a comment - Integrated in ZooKeeper-trunk #558 (See http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/558/ )
          Hide
          Benjamin Reed added a comment -

          Committed revision 884554.
          thank you david!

          Show
          Benjamin Reed added a comment - Committed revision 884554. thank you david!
          Hide
          David Bosschaert added a comment -

          Hi Patrick,

          You can already use this as-is in Felix or Equinox (or any other OSGi container). The value that the metadata in the buildxml.patch provides is that it exports (and imports) packages the OSGi way, which means that you can build a zookeeper client in its own bundle that pulls in the zookeeper classes using OSGi dependency management.

          Here's a quick example of how to use it (using Felix 2.0.0):

          .../felix-framework-2.0.0>java -jar bin/felix.jar
          
          Welcome to Felix
          ================
          
          -> start http://www.springsource.com/repository/app/bundle/version/download?name=com.springsource.org.apache.log4j&version=1.2.15&type=binary
          Bundle ID: 4
          -> start file:///.../zookeeper/build/zookeeper-3.3.0.jar
          -> ps
          START LEVEL 1
             ID   State         Level  Name
          [   0] [Active     ] [    0] System Bundle (2.0.0)
          [   1] [Active     ] [    1] Apache Felix Bundle Repository (1.4.1)
          [   2] [Active     ] [    1] Apache Felix Shell Service (1.4.0)
          [   3] [Active     ] [    1] Apache Felix Shell TUI (1.4.0)
          [   4] [Active     ] [    1] Apache Log4J (1.2.15)
          [   5] [Active     ] [    1] ZooKeeper Bundle (3.3.0)
          
          -> inspect package requirement 5
          org.apache.hadoop.zookeeper [5] imports packages:
          -------------------------------------------------
          org.osgi.framework; version=1.5.0 -> org.apache.felix.framework [0]
          org.osgi.util.tracker; version=1.4.0 -> org.apache.felix.framework [0]
          javax.management; version=1.6.0 -> org.apache.felix.framework [0]
          org.apache.log4j; version=1.2.15 -> com.springsource.org.apache.log4j [4]
          
          -> inspect package capability 5
          org.apache.hadoop.zookeeper [5] exports packages:
          -------------------------------------------------
          org.apache.zookeeper; version=3.3.0
          org.apache.zookeeper.data; version=3.3.0
          org.apache.zookeeper.version; version=3.3.0
          org.apache.zookeeper.server; version=3.3.0
          org.apache.zookeeper.server.quorum; version=3.3.0
          

          As you can see the Log4J dependency is provided through a Log4J bundle. I'm using one straight from the SpringSource bundle repository (the SpringSource guys have turned many libraries into OSGi bundles), but you may also be able to find a Log4J bundle elsewhere.
          After installing the Zookeeper Bundle (the jar is now also an OSGi bundle) you can see that it uses OSGi imports to satisfy its dependencies (e.g. it links to bundle 4 for its log4j imports) and it exports the zookeeper packages. The way to use them in OSGi-based clients is simply through imports (as here), but additionally the client bundle will have Import-Package statements for the zookeeper packages in its manifest.

          Show
          David Bosschaert added a comment - Hi Patrick, You can already use this as-is in Felix or Equinox (or any other OSGi container). The value that the metadata in the buildxml.patch provides is that it exports (and imports) packages the OSGi way, which means that you can build a zookeeper client in its own bundle that pulls in the zookeeper classes using OSGi dependency management. Here's a quick example of how to use it (using Felix 2.0.0): .../felix-framework-2.0.0>java -jar bin/felix.jar Welcome to Felix ================ -> start http: //www.springsource.com/repository/app/bundle/version/download?name=com.springsource.org.apache.log4j&version=1.2.15&type=binary Bundle ID: 4 -> start file: ///.../zookeeper/build/zookeeper-3.3.0.jar -> ps START LEVEL 1 ID State Level Name [ 0] [Active ] [ 0] System Bundle (2.0.0) [ 1] [Active ] [ 1] Apache Felix Bundle Repository (1.4.1) [ 2] [Active ] [ 1] Apache Felix Shell Service (1.4.0) [ 3] [Active ] [ 1] Apache Felix Shell TUI (1.4.0) [ 4] [Active ] [ 1] Apache Log4J (1.2.15) [ 5] [Active ] [ 1] ZooKeeper Bundle (3.3.0) -> inspect package requirement 5 org.apache.hadoop.zookeeper [5] imports packages: ------------------------------------------------- org.osgi.framework; version=1.5.0 -> org.apache.felix.framework [0] org.osgi.util.tracker; version=1.4.0 -> org.apache.felix.framework [0] javax.management; version=1.6.0 -> org.apache.felix.framework [0] org.apache.log4j; version=1.2.15 -> com.springsource.org.apache.log4j [4] -> inspect package capability 5 org.apache.hadoop.zookeeper [5] exports packages: ------------------------------------------------- org.apache.zookeeper; version=3.3.0 org.apache.zookeeper.data; version=3.3.0 org.apache.zookeeper.version; version=3.3.0 org.apache.zookeeper.server; version=3.3.0 org.apache.zookeeper.server.quorum; version=3.3.0 As you can see the Log4J dependency is provided through a Log4J bundle. I'm using one straight from the SpringSource bundle repository (the SpringSource guys have turned many libraries into OSGi bundles), but you may also be able to find a Log4J bundle elsewhere. After installing the Zookeeper Bundle (the jar is now also an OSGi bundle) you can see that it uses OSGi imports to satisfy its dependencies (e.g. it links to bundle 4 for its log4j imports) and it exports the zookeeper packages. The way to use them in OSGi-based clients is simply through imports (as here ), but additionally the client bundle will have Import-Package statements for the zookeeper packages in its manifest.
          Hide
          Benjamin Reed added a comment -

          yes just the buildxml.patch, the service instance will be registered in ZOOKEEPER-584 and we will need to pull in felix for that test. i would think that the activator for ZOOKEEPER-584 will go into a contrib and will will build its own little jar file. that way we only need to pull in felix for the contrib not for the whole of zookeeper.

          Show
          Benjamin Reed added a comment - yes just the buildxml.patch, the service instance will be registered in ZOOKEEPER-584 and we will need to pull in felix for that test. i would think that the activator for ZOOKEEPER-584 will go into a contrib and will will build its own little jar file. that way we only need to pull in felix for the contrib not for the whole of zookeeper.
          Hide
          Patrick Hunt added a comment -

          Just to summarize - what's being applied here (patch)? Just buildxmlpatch.patch and nothing else (the other attachments are obsolete?)

          I agree that no new test is necessary, but it would be interesting to know how to try this out. Is there a way to load this into something like Apache Felix yet, or does that have to wait for the follow-on jira? (I would suggest - please create one and link it to this jira)

          Show
          Patrick Hunt added a comment - Just to summarize - what's being applied here (patch)? Just buildxmlpatch.patch and nothing else (the other attachments are obsolete?) I agree that no new test is necessary, but it would be interesting to know how to try this out. Is there a way to load this into something like Apache Felix yet, or does that have to wait for the follow-on jira? (I would suggest - please create one and link it to this jira)
          Hide
          Benjamin Reed added a comment -

          +1 to commit. no need for tests in this case. once we do the activator we will need test for that which will also cover the meta data as well. (the automatic testing is cool though isn't it?) unless there are any objections i think this is ready to commit.

          Show
          Benjamin Reed added a comment - +1 to commit. no need for tests in this case. once we do the activator we will need test for that which will also cover the meta data as well. (the automatic testing is cool though isn't it?) unless there are any objections i think this is ready to commit.
          Hide
          Alan Cabrera added a comment -

          Why not use PAX Exam?

          Show
          Alan Cabrera added a comment - Why not use PAX Exam?
          Hide
          David Bosschaert added a comment -

          Wow - that's excellent! An automated patch review process!
          This patch only contains additional attributes to the zookeeper.jar file. The best way to write tests for them will be as part of a system test that's using the jar as a bundle in a real OSGi container. Does the zookeeper infrastructure have such an area?

          Another way to write tests would simply be to check that the headers are there. This would be easier to do, but it would require that the tests can expect the zookeeper.jar file in a well-known location. For this the jar has to be created before those tests are run. Is there such a phase in the zookeeper build system?

          Show
          David Bosschaert added a comment - Wow - that's excellent! An automated patch review process! This patch only contains additional attributes to the zookeeper.jar file. The best way to write tests for them will be as part of a system test that's using the jar as a bundle in a real OSGi container. Does the zookeeper infrastructure have such an area? Another way to write tests would simply be to check that the headers are there. This would be easier to do, but it would require that the tests can expect the zookeeper.jar file in a well-known location. For this the jar has to be created before those tests are run. Is there such a phase in the zookeeper build system?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12425830/buildxmlpatch.patch
          against trunk revision 882744.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/74/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/74/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/74/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12425830/buildxmlpatch.patch against trunk revision 882744. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/74/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/74/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-h8.grid.sp2.yahoo.net/74/console This message is automatically generated.
          Hide
          David Bosschaert added a comment -

          Yes, I agree that it would be better to put uses clauses on the Export-Package declaration. However, you really need a tool such as BND (http://www.aqute.biz/Code/Bnd) (or the maven-bundle-plugin which uses bnd) to generate these for you. Putting them in by hand is hard, error prone and difficult to maintain (IMHO).

          I already mention BND in the patch and would be happy to update it to get ant to use BND if people think this would be good. We can then let BND generate the uses clauses for us.

          Show
          David Bosschaert added a comment - Yes, I agree that it would be better to put uses clauses on the Export-Package declaration. However, you really need a tool such as BND ( http://www.aqute.biz/Code/Bnd ) (or the maven-bundle-plugin which uses bnd) to generate these for you. Putting them in by hand is hard, error prone and difficult to maintain (IMHO). I already mention BND in the patch and would be happy to update it to get ant to use BND if people think this would be good. We can then let BND generate the uses clauses for us.
          Hide
          Alan Cabrera added a comment -

          Just curious, no "uses" clauses?

          Show
          Alan Cabrera added a comment - Just curious, no "uses" clauses?
          Hide
          David Bosschaert added a comment -

          The patch named 'buildxmlpatch.patch' contains changes to the build.xml to cover case 1.

          It does an export-package of
          org.apache.zookeeper.server
          org.apache.zookeeper.server.quorum
          to make it possible to launch the server process from a separate bundle.

          If the general feeling is that the Zookeeper codebase isn't ready for case 3, I'm happy to host my Activator/ManageService elsewhere until it is ready for it.

          Show
          David Bosschaert added a comment - The patch named 'buildxmlpatch.patch' contains changes to the build.xml to cover case 1. It does an export-package of org.apache.zookeeper.server org.apache.zookeeper.server.quorum to make it possible to launch the server process from a separate bundle. If the general feeling is that the Zookeeper codebase isn't ready for case 3, I'm happy to host my Activator/ManageService elsewhere until it is ready for it.
          Hide
          David Bosschaert added a comment -

          Let me explain the context of my work. For the OSGi Remote Services Admin Service (an OSGi specification based on the Distributed OSGi RFC 119 work) we are working on the Reference Implementation in CXF (http://cxf.apache.org/distributed-osgi.html). The Reference Implementation supports the Discovery concepts of the Remote Service specification and for the implementation we make use of ZooKeeper.

          In our code, we use zookeeper in two ways:

          • In zookeeper client code. This code resides in bundles that import the org.apache.zookeeper and org.apache.zookeeper.data
          • We also need to be able to run the zookeeper server from inside an OSGi bundle

          So this is really cases 1 & 3 above.

          I think we all agree on 1, and I can certainly put that in a separate patch.

          On part 2 - this is the one that Alan filed ZOOKEEPER-584 for, right? This is a case that I don't need at this point, and therefore I probably don't understand what the best way to realize it is. I understand that this can be quite useful and registering a 'trivial' zookeeper object in the service registry is easy, but I guess what Alan is really looking for is an object from which he can get some management information. I really see this as a separate issue and one possibly best solved by someone who actually has a need for this.

          The code that I attached does address part 3 somewhat, although I agree that it could be better. However it's a start. The zookeeper server starts and can be configured. I can try to make the shutdown/restart on reconfiguration work as well. I haven't been troubled by the System.exits() yet. It works well enough for me at the moment, although it could be improved. Could we not do the improvement over time in the future, starting off with something that works a bit like in my patch and then making it better over time?

          Patrick, on your comment re a contrib package. First of all the code is not specific to an osgi container and should work equally in all osgi frameworks. Do you mean just putting the source code in a separate location or are you also thinking of putting it in a separate jar file? I would suggest putting the Activator and ManagedService in the ordinary zookeeper.jar. Outside of an OSGi framework these classes are simply ignored, but when the jar is used as an OSGi bundle they are automatically found and used. If you put it in a separate jar you will have to deploy 2 bundles before you can run the zookeeper server in an OSGi framework, which is possible but you'd have to export org.apache.zookeeper.server which is currently internal to make it work. Besides these two classes are currently only about 5kb in total.

          Show
          David Bosschaert added a comment - Let me explain the context of my work. For the OSGi Remote Services Admin Service (an OSGi specification based on the Distributed OSGi RFC 119 work) we are working on the Reference Implementation in CXF ( http://cxf.apache.org/distributed-osgi.html ). The Reference Implementation supports the Discovery concepts of the Remote Service specification and for the implementation we make use of ZooKeeper. In our code, we use zookeeper in two ways: In zookeeper client code. This code resides in bundles that import the org.apache.zookeeper and org.apache.zookeeper.data We also need to be able to run the zookeeper server from inside an OSGi bundle So this is really cases 1 & 3 above. I think we all agree on 1, and I can certainly put that in a separate patch. On part 2 - this is the one that Alan filed ZOOKEEPER-584 for, right? This is a case that I don't need at this point, and therefore I probably don't understand what the best way to realize it is. I understand that this can be quite useful and registering a 'trivial' zookeeper object in the service registry is easy, but I guess what Alan is really looking for is an object from which he can get some management information. I really see this as a separate issue and one possibly best solved by someone who actually has a need for this. The code that I attached does address part 3 somewhat, although I agree that it could be better. However it's a start. The zookeeper server starts and can be configured. I can try to make the shutdown/restart on reconfiguration work as well. I haven't been troubled by the System.exits() yet. It works well enough for me at the moment, although it could be improved. Could we not do the improvement over time in the future, starting off with something that works a bit like in my patch and then making it better over time? Patrick, on your comment re a contrib package. First of all the code is not specific to an osgi container and should work equally in all osgi frameworks. Do you mean just putting the source code in a separate location or are you also thinking of putting it in a separate jar file? I would suggest putting the Activator and ManagedService in the ordinary zookeeper.jar. Outside of an OSGi framework these classes are simply ignored, but when the jar is used as an OSGi bundle they are automatically found and used. If you put it in a separate jar you will have to deploy 2 bundles before you can run the zookeeper server in an OSGi framework, which is possible but you'd have to export org.apache.zookeeper.server which is currently internal to make it work. Besides these two classes are currently only about 5kb in total.
          Hide
          Patrick Hunt added a comment -

          1) would go into build.xml

          2) & 3) would be in a contrib package, correct? src/contrib/osgi – or is this stuff typically implementation
          specific such as src/contrib/felix ? (assuming not the latter)

          Show
          Patrick Hunt added a comment - 1) would go into build.xml 2) & 3) would be in a contrib package, correct? src/contrib/osgi – or is this stuff typically implementation specific such as src/contrib/felix ? (assuming not the latter)
          Hide
          Benjamin Reed added a comment -

          david, nice work. i think we should break this patch into possibly three parts:

          1) manifest update to just export the right packages
          2) an activator that registers a ZooKeeper object as a service. (this would be a client object)
          3) an activator that starts up a server.

          you seem to be ignoring 2). it might be useful to have the bundles of an osgi framework share a ZooKeeper object. maybe not though. sharing makes it nice to get a preconfigured connected ZooKeeper object from the service registry.

          we can't really support 3) properly right now. it is possible to shutdown a server and start it back up. the interfaces aren't very well exposed but the tests do it. a bigger problem is that we have System.exits in our code, and even if the bundle doesn't have permission to call System.exit, not exiting can cause bad things to happen.

          i would suggest focusing this issue on 1) and possibly 2), but leave 3) for a separate issue.

          Show
          Benjamin Reed added a comment - david, nice work. i think we should break this patch into possibly three parts: 1) manifest update to just export the right packages 2) an activator that registers a ZooKeeper object as a service. (this would be a client object) 3) an activator that starts up a server. you seem to be ignoring 2). it might be useful to have the bundles of an osgi framework share a ZooKeeper object. maybe not though. sharing makes it nice to get a preconfigured connected ZooKeeper object from the service registry. we can't really support 3) properly right now. it is possible to shutdown a server and start it back up. the interfaces aren't very well exposed but the tests do it. a bigger problem is that we have System.exits in our code, and even if the bundle doesn't have permission to call System.exit, not exiting can cause bad things to happen. i would suggest focusing this issue on 1) and possibly 2), but leave 3) for a separate issue.
          Hide
          Alan Cabrera added a comment -

          Sure. I saw that we were adding a bundle activator and deployment via Configuration Admin Service so I wasn't sure where we were drawing the line here. Issue added.

          Show
          Alan Cabrera added a comment - Sure. I saw that we were adding a bundle activator and deployment via Configuration Admin Service so I wasn't sure where we were drawing the line here. Issue added.
          Hide
          David Bosschaert added a comment -

          Hi Alan, that sounds like a useful idea to me and using a service in the OSGi Service Registry sounds like a really good approach but I personally think that this should be a separate bug/enhancement. Let's get the basics working first

          Show
          David Bosschaert added a comment - Hi Alan, that sounds like a useful idea to me and using a service in the OSGi Service Registry sounds like a really good approach but I personally think that this should be a separate bug/enhancement. Let's get the basics working first
          Hide
          Alan Cabrera added a comment -

          I would like to be informed when the Zookeeper service is started/stopped in the OSGi framework so that I can perform some "extender" like tasks in tandem. Would it be a good idea to publish a management interface to the service registry when Zookeeper is started by the ManagedService?

          Show
          Alan Cabrera added a comment - I would like to be informed when the Zookeeper service is started/stopped in the OSGi framework so that I can perform some "extender" like tasks in tandem. Would it be a good idea to publish a management interface to the service registry when Zookeeper is started by the ManagedService ?
          Hide
          David Bosschaert added a comment -

          The attached patch (named zk_patch3.patch) implements some of the functionality. Its not fully finished yet, I'm attaching it at this point to get some feedback.

          The patch:

          • Adds the OSGi Metadata to the MANIFEST.MF
          • Can be used as just a library bundle if you want to write a zookeeper client in another OSGi bundle.
          • Can be used to run the zookeeper server. To do this the server has to be configured. Configuration is handled through the OSGi Configuration Admin service. The bundle has an optional dependency on this service and registers an org.osgi.service.cm.ManagedService service to be configured by CM. The PID for this service is org.apache.zookeeper and all of the configuration variables are the same as in the non-OSGi zoo.cfg file (it uses the same ServerConfig class to handle them).
            All configuration properties have defaults provided, except for the 'clientPort' one, which has to be provided through Config Admin. For the data directory the OSGi Bundle-private storage directory is used by default.
          • The jar has an OSGi BundleActivator class, which will be triggered if run on an OSGi framework.
          • BTW the zookeeper.jar stil also works as a POJ (plain old Jar

          note that I updated the ivy.xml file to pull in two OSGi jars that are needed at build time. However they don't need to be redistributed with the zookeeper jar as they are provided by the OSGi framework.

          There are still a few open questions that I have:

          • In OSGi things should really be (re)configurable at runtime. This means that the Configuration Admin Service may call your ManagedService.updated() callback with changed properties at any time. I guess the easiest (but not necessarily the most elegant) way to handle this is by taking down the ZooKeeperServerMain and relaunching it with the modified properties. ZooKeeperServerMain has a shutdown() method that I could use for this. Would that be the best idea?
          • I'm not (yet) familiar with the cluster setup and am wondering whether the configuration approach that I took also works in that case.
            I also intend to provide some unit tests before closing this off.

          BTW I haven't gone into detail how the OSGi Configuration Admin service typically works. Let me know if I need to provide more info on that.

          Show
          David Bosschaert added a comment - The attached patch (named zk_patch3.patch) implements some of the functionality. Its not fully finished yet, I'm attaching it at this point to get some feedback. The patch: Adds the OSGi Metadata to the MANIFEST.MF Can be used as just a library bundle if you want to write a zookeeper client in another OSGi bundle. Can be used to run the zookeeper server. To do this the server has to be configured. Configuration is handled through the OSGi Configuration Admin service. The bundle has an optional dependency on this service and registers an org.osgi.service.cm.ManagedService service to be configured by CM. The PID for this service is org.apache.zookeeper and all of the configuration variables are the same as in the non-OSGi zoo.cfg file (it uses the same ServerConfig class to handle them). All configuration properties have defaults provided, except for the 'clientPort' one, which has to be provided through Config Admin. For the data directory the OSGi Bundle-private storage directory is used by default. The jar has an OSGi BundleActivator class, which will be triggered if run on an OSGi framework. BTW the zookeeper.jar stil also works as a POJ (plain old Jar note that I updated the ivy.xml file to pull in two OSGi jars that are needed at build time. However they don't need to be redistributed with the zookeeper jar as they are provided by the OSGi framework. There are still a few open questions that I have: In OSGi things should really be (re)configurable at runtime. This means that the Configuration Admin Service may call your ManagedService.updated() callback with changed properties at any time. I guess the easiest (but not necessarily the most elegant) way to handle this is by taking down the ZooKeeperServerMain and relaunching it with the modified properties. ZooKeeperServerMain has a shutdown() method that I could use for this. Would that be the best idea? I'm not (yet) familiar with the cluster setup and am wondering whether the configuration approach that I took also works in that case. I also intend to provide some unit tests before closing this off. BTW I haven't gone into detail how the OSGi Configuration Admin service typically works. Let me know if I need to provide more info on that.
          Hide
          David Bosschaert added a comment -

          Hi Benjamin, that helps. I'll play around with that explicit subset.
          What I really need is both. Provide client access to the Zookeeper API, but also a way to start the server. To do that we may not have to expose those server classes at all. One thing we could do is add an activator to the zookeeper jar and have it use the OSGI Configuration Admin to control the zookeeper server. It should be possible to do this without changing it being a plain jar too.
          Anyway - I'll try to find some time over the next while to play with this and will let you know how far I get.

          Show
          David Bosschaert added a comment - Hi Benjamin, that helps. I'll play around with that explicit subset. What I really need is both. Provide client access to the Zookeeper API, but also a way to start the server. To do that we may not have to expose those server classes at all. One thing we could do is add an activator to the zookeeper jar and have it use the OSGI Configuration Admin to control the zookeeper server. It should be possible to do this without changing it being a plain jar too. Anyway - I'll try to find some time over the next while to play with this and will let you know how far I get.
          Hide
          Benjamin Reed added a comment -

          oh sorry david. so i still have the same concern with the full manifest.mf, but before that i was wondering: are you trying to provide the bundle so that other bundles can use zookeeper or so that the bundle can start up a zookeeper server?

          most of the packages imported and exported are internal to zookeeper and should be kept private. if we want to just provide access to the client API we should just list org.apache.zookeeper and org.apache.zookeeper.data (possibly org.apache.zookeeper.version). we should also use the script to set the version rather than hard code it. if you want to start the server, we should really have a separate package with just the classes/interfaces needed to manage a server instance and export that.

          the only import we need is log4j. is there already a standard log4j bundle?

          Show
          Benjamin Reed added a comment - oh sorry david. so i still have the same concern with the full manifest.mf, but before that i was wondering: are you trying to provide the bundle so that other bundles can use zookeeper or so that the bundle can start up a zookeeper server? most of the packages imported and exported are internal to zookeeper and should be kept private. if we want to just provide access to the client API we should just list org.apache.zookeeper and org.apache.zookeeper.data (possibly org.apache.zookeeper.version). we should also use the script to set the version rather than hard code it. if you want to start the server, we should really have a separate package with just the classes/interfaces needed to manage a server instance and export that. the only import we need is log4j. is there already a standard log4j bundle?
          Hide
          Benjamin Reed added a comment -

          right these are osgi specific tags that will get ignored normally.

          Show
          Benjamin Reed added a comment - right these are osgi specific tags that will get ignored normally.
          Hide
          Patrick Hunt added a comment -

          A dumb q: adding this to the manifest of the zk jar will have no effect on non-osgi containers (etc...) correct?

          Also, re the original question "Is it really necessary to put the source code in the Jar file too":
          notice that in the trunk we have changed things a bit since this jira was created. We still have
          the original jar which includes sources, but we also have an additional binary only jar (class files) in addition
          to separate source and javadoc jars. You will see this in the "package" target of the latest build.xml
          This was added for Maven – we should be sure to include the manifest changes to those jars
          (just the jars containing class files I guess?)

          Show
          Patrick Hunt added a comment - A dumb q: adding this to the manifest of the zk jar will have no effect on non-osgi containers (etc...) correct? Also, re the original question "Is it really necessary to put the source code in the Jar file too": notice that in the trunk we have changed things a bit since this jira was created. We still have the original jar which includes sources, but we also have an additional binary only jar (class files) in addition to separate source and javadoc jars. You will see this in the "package" target of the latest build.xml This was added for Maven – we should be sure to include the manifest changes to those jars (just the jars containing class files I guess?)
          Hide
          David Bosschaert added a comment -

          Hi Benjamin. Did you actually take a look at the attached MANIFEST.MF? Are the import and export statements in there correct? What is in the XML above is just the maven-bundle-plugin/BND instructions to create the exact list by introspection
          If you think it would be better to explicitly list everything when creating the manifest, please state what you would like to see in there.

          Show
          David Bosschaert added a comment - Hi Benjamin. Did you actually take a look at the attached MANIFEST.MF? Are the import and export statements in there correct? What is in the XML above is just the maven-bundle-plugin/BND instructions to create the exact list by introspection If you think it would be better to explicitly list everything when creating the manifest, please state what you would like to see in there.
          Hide
          Benjamin Reed added a comment -

          sorry i didn't notice this sooner. this is a great idea, and certainly reasonable. i think the import and export packages statement is incorrect. we should list the exact dependencies.

          Show
          Benjamin Reed added a comment - sorry i didn't notice this sooner. this is a great idea, and certainly reasonable. i think the import and export packages statement is incorrect. we should list the exact dependencies.
          Hide
          David Bosschaert added a comment -

          I'll also have a look at how to run the zookeeper server in OSGi.

          Show
          David Bosschaert added a comment - I'll also have a look at how to run the zookeeper server in OSGi.
          Hide
          David Bosschaert added a comment -

          The manifest generated by the CXF/DOSGi build system.

          Show
          David Bosschaert added a comment - The manifest generated by the CXF/DOSGi build system.

            People

            • Assignee:
              David Bosschaert
              Reporter:
              David Bosschaert
            • Votes:
              6 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development