Uploaded image for project: 'OFBiz'
  1. OFBiz
  2. OFBIZ-7972

Create a (PoC) plugin system for OFBiz based on Gradle

    Details

      Description

      This JIRA houses the attempt to create a plugin system for OFBiz based on gradle. The plugin system should have the following features:

      • It is independent of the version control system, file system, operating system and any other environment tools surrounding OFBiz. Instead, it should be fully integrated with OFBiz.
      • It should unify the way OFBiz is extended, both for official supported plugins and for third party plugins
      • The plugin system takes advantage of the concept of a "component" in OFBiz and utilize it as the building block for plugins
      • The plugin system supports the concepts of "Repository" and "Dependency Management". If possible, the plugin system will utilize these concepts directly from Gradle instead of writing them from scratch.

      The initial API for the plugin system will be small and expands as needed. The API should include the following gradle tasks:

      • createPlugin: creates a new plugin based on templates and place it in the plugins directory
      • installPlugin: Activate a plugin and run Install. Plugin must exist in /specialpurpose
      • uninstallPlugin: Deactivate a plugin and run its uninstall script
      • publishPlugin: publish a plugin to local maven repository
      • pullPlugin: download a plugin with its dependencies (plugins) from a repository and install them
      1. OFBIZ-7972.patch
        24 kB
        Taher Alkhateeb
      2. OFBIZ-7972.patch
        26 kB
        Taher Alkhateeb
      3. OFBIZ-7972.patch
        27 kB
        Taher Alkhateeb

        Issue Links

          Activity

          Hide
          taher Taher Alkhateeb added a comment -

          I made a comment on the ML in a new design utilizing maven in this thread

          Also, this github repo contains useful resources on publishing to bintray (jcenter)

          Show
          taher Taher Alkhateeb added a comment - I made a comment on the ML in a new design utilizing maven in this thread Also, this github repo contains useful resources on publishing to bintray (jcenter)
          Hide
          taher Taher Alkhateeb added a comment -

          This is an intial PoC patch that shows the plugin system. All help appreciated in testing and providing feedback. To test you can apply the following steps:

          ./gradlew createPlugin -PpluginId=myplugin
          ./gradlew publishPlugin -PpluginId=myplugin
          rm -Rf specialpurpose/myplugin
          svn revert specialpurpose/component-load.xml
          ./gradlew pullPlugin -PdependencyId=org.apache.ofbiz.plugin:myplugin:0.1.0-SNAPSHOT
          

          I have also provided documentation in README.md for the plugin commands. And naturally, this removes the "createComponent" task since it will be replaced by the plugin system

          Show
          taher Taher Alkhateeb added a comment - This is an intial PoC patch that shows the plugin system. All help appreciated in testing and providing feedback. To test you can apply the following steps: ./gradlew createPlugin -PpluginId=myplugin ./gradlew publishPlugin -PpluginId=myplugin rm -Rf specialpurpose/myplugin svn revert specialpurpose/component-load.xml ./gradlew pullPlugin -PdependencyId=org.apache.ofbiz.plugin:myplugin:0.1.0-SNAPSHOT I have also provided documentation in README.md for the plugin commands. And naturally, this removes the "createComponent" task since it will be replaced by the plugin system
          Hide
          gil portenseigne Gil Portenseigne added a comment -

          Hi Taher, great job so far,
          I did easily managed to create my own plugin, publish it to my local repos, uninstall it and pull it from my local repo.

          The README is pretty clear (only copy/paste issue line 496).

          I feel that a removePlugin that will be a shortcut to uninstallPlugin + rm -Rf speciapurpose/myPluginId, could be good. But while writing it, i'm not sure about me yet... Food for thought

          These for basic test feedback, I continue my tests !

          Show
          gil portenseigne Gil Portenseigne added a comment - Hi Taher, great job so far, I did easily managed to create my own plugin, publish it to my local repos, uninstall it and pull it from my local repo. The README is pretty clear (only copy/paste issue line 496). I feel that a removePlugin that will be a shortcut to uninstallPlugin + rm -Rf speciapurpose/myPluginId, could be good. But while writing it, i'm not sure about me yet... Food for thought These for basic test feedback, I continue my tests !
          Hide
          taher Taher Alkhateeb added a comment -

          Hi Gil, thank you I hesitated in making the uninstallPlugin task delete the folder, and I like your idea of remove plugin being equal to uninstall+delete. Maybe we should call it deletePlugin because there is a function in the script called removePlugin that removes it from component-load.xml.

          I also discovered a few more typos in README.md so I'll fix those in addition to the copy-paste one you caught. I'll patch again soon.

          There is a lot more to discuss but I'll wait for you to explore some more. Thank you for all the help.

          Show
          taher Taher Alkhateeb added a comment - Hi Gil, thank you I hesitated in making the uninstallPlugin task delete the folder, and I like your idea of remove plugin being equal to uninstall+delete. Maybe we should call it deletePlugin because there is a function in the script called removePlugin that removes it from component-load.xml. I also discovered a few more typos in README.md so I'll fix those in addition to the copy-paste one you caught. I'll patch again soon. There is a lot more to discuss but I'll wait for you to explore some more. Thank you for all the help.
          Hide
          gil portenseigne Gil Portenseigne added a comment -

          Another thing i didn't react to when coding createComponent task was done is to have resourceName capitalised if not given in tasks parameters, to avoid myComponentScreens.xml instead of MyComponentScreens.xml...

          Show
          gil portenseigne Gil Portenseigne added a comment - Another thing i didn't react to when coding createComponent task was done is to have resourceName capitalised if not given in tasks parameters, to avoid myComponentScreens.xml instead of MyComponentScreens.xml...
          Hide
          mbrohl Michael Brohl added a comment -

          Hi Taher,

          many thanks for your efforts! I have tested all described steps and everything worked very well.
          Great documentation apart from the quirks already mentioned.

          I would also be in favor of a removePlugin task that does uninstall and delete. I'd suggest to rename the existing removePlugin to deactivatePlugin, this would better describe what the task really does.

          One idea comes to mind: is it planned to have some version checking mechanism? If I want to pull/install a remote plugin it would be nice if the installer would check if the plugin is compatible with the base system.

          I will play with the plugin system in the next days and report any findings or ideas.

          Thanks for your great work!

          Show
          mbrohl Michael Brohl added a comment - Hi Taher, many thanks for your efforts! I have tested all described steps and everything worked very well. Great documentation apart from the quirks already mentioned. I would also be in favor of a removePlugin task that does uninstall and delete. I'd suggest to rename the existing removePlugin to deactivatePlugin, this would better describe what the task really does. One idea comes to mind: is it planned to have some version checking mechanism? If I want to pull/install a remote plugin it would be nice if the installer would check if the plugin is compatible with the base system. I will play with the plugin system in the next days and report any findings or ideas. Thanks for your great work!
          Hide
          taher Taher Alkhateeb added a comment -

          Hi Michael, thank you for the help, I will update the patch soon with your feedback as well as that of Gil's

          You asked me a difficult question on versioning which I was thinking about a lot. In my current design I made OFBiz plugins themselves versioned using Semantic Versioning 2.0.0 and you can have a very complex dependency tree with versions without a problem and it will all be resolved automatically. This is just good old maven combined with the power of Gradle that makes this whole system seamless.

          OFBiz itself, however, is not versioned, so I do not have at the moment a solution to say that this plugin only works with OFBiz 16 and that plugin works with 15 and so on. The reasons for this problem:

          • There is no file in OFBiz that tells us what is the current version of OFBiz
          • Using subversion to keep track of versions in my opinion is a bad idea because it makes the solution brittle and dependent on tools instead of being part of the OFBiz solution. If we switch the version control system in the future then our solution breaks down.
          • What about trunk? It's hard to make plugins dependent on trunk because trunk is this constantly changing thing.

          So, I have no clear solution in my mind at the moment, but I am sharing my ideas to brainstorm together

          Show
          taher Taher Alkhateeb added a comment - Hi Michael, thank you for the help, I will update the patch soon with your feedback as well as that of Gil's You asked me a difficult question on versioning which I was thinking about a lot. In my current design I made OFBiz plugins themselves versioned using Semantic Versioning 2.0.0 and you can have a very complex dependency tree with versions without a problem and it will all be resolved automatically. This is just good old maven combined with the power of Gradle that makes this whole system seamless. OFBiz itself, however, is not versioned, so I do not have at the moment a solution to say that this plugin only works with OFBiz 16 and that plugin works with 15 and so on. The reasons for this problem: There is no file in OFBiz that tells us what is the current version of OFBiz Using subversion to keep track of versions in my opinion is a bad idea because it makes the solution brittle and dependent on tools instead of being part of the OFBiz solution. If we switch the version control system in the future then our solution breaks down. What about trunk? It's hard to make plugins dependent on trunk because trunk is this constantly changing thing. So, I have no clear solution in my mind at the moment, but I am sharing my ideas to brainstorm together
          Hide
          taher Taher Alkhateeb added a comment - - edited

          Okay this new patch delivers the following:

          • Fix the documentation in README.md
          • Renamed removePlugin and addPlugin to deactivatePlugin and activatePlugin as per Michael's suggestion
          • Introduce a task called removePlugin that in addition to uninstalling a plugin also deletes it from the filesystem as recommended by Michael and Gil.
          • Make the componentResourceName capitalized (Thank you Gil for the code snippet)
          Show
          taher Taher Alkhateeb added a comment - - edited Okay this new patch delivers the following: Fix the documentation in README.md Renamed removePlugin and addPlugin to deactivatePlugin and activatePlugin as per Michael's suggestion Introduce a task called removePlugin that in addition to uninstalling a plugin also deletes it from the filesystem as recommended by Michael and Gil. Make the componentResourceName capitalized (Thank you Gil for the code snippet)
          Hide
          taher Taher Alkhateeb added a comment -

          Newest version attached, minor formatting and improvements in documentation

          Show
          taher Taher Alkhateeb added a comment - Newest version attached, minor formatting and improvements in documentation
          Hide
          jacopoc Jacopo Cappellato added a comment -

          It looks a great start to me.

          One minor question: are both entries required?

          +    junitReport 'org.apache.ant:ant-junit:1.9.7'
          +    junitReport 'org.apache.ant:ant-junit4:1.9.7'
          

          I have voted for this ticket and I would be in favor of committing it into trunk, maybe mentioning (at the beginning of the README file) that is experimental.

          Show
          jacopoc Jacopo Cappellato added a comment - It looks a great start to me. One minor question: are both entries required? + junitReport 'org.apache.ant:ant-junit:1.9.7' + junitReport 'org.apache.ant:ant-junit4:1.9.7' I have voted for this ticket and I would be in favor of committing it into trunk, maybe mentioning (at the beginning of the README file) that is experimental.
          Hide
          mbrohl Michael Brohl added a comment -

          +1 for putting it into trunk.

          That would make it a lot easier to test & play instead of applying patches and reverting.

          Maybe we should keep the "createComponent" task as long as it is experimental then?

          Show
          mbrohl Michael Brohl added a comment - +1 for putting it into trunk. That would make it a lot easier to test & play instead of applying patches and reverting. Maybe we should keep the "createComponent" task as long as it is experimental then?
          Hide
          gil portenseigne Gil Portenseigne added a comment -

          +1, no problem with removing createComponent in my opinion, since createPlugin do the same.

          Show
          gil portenseigne Gil Portenseigne added a comment - +1, no problem with removing createComponent in my opinion, since createPlugin do the same.
          Hide
          taher Taher Alkhateeb added a comment -

          Good catch Jacopo, yeah I deleted 'org.apache.ant:ant-junit4:1.9.7' as it is duplicated and the system works without it.

          Thank you Jacopo, Michael and Gil for the vote to push the experimental code to Trunk. I'll wait for a while for feedback from others before proceeding.

          Show
          taher Taher Alkhateeb added a comment - Good catch Jacopo, yeah I deleted 'org.apache.ant:ant-junit4:1.9.7' as it is duplicated and the system works without it. Thank you Jacopo, Michael and Gil for the vote to push the experimental code to Trunk. I'll wait for a while for feedback from others before proceeding.
          Hide
          taher Taher Alkhateeb added a comment -

          Hi Folks, I just had a new idea before committing. I would like to rename publishPlugin to pushPlugin. This way we have a nice consistent naming i.e. pulling from or pushing to a maven Repository. WDYT

          Show
          taher Taher Alkhateeb added a comment - Hi Folks, I just had a new idea before committing. I would like to rename publishPlugin to pushPlugin. This way we have a nice consistent naming i.e. pulling from or pushing to a maven Repository. WDYT
          Hide
          jacopoc Jacopo Cappellato added a comment -

          I like it.
          I am looking forward at seeing this code in our repo: after that it will be easier to test it and paly with it and gather further feedback and enhancements from the community.

          Show
          jacopoc Jacopo Cappellato added a comment - I like it. I am looking forward at seeing this code in our repo: after that it will be easier to test it and paly with it and gather further feedback and enhancements from the community.
          Hide
          mbrohl Michael Brohl added a comment -

          That would be ok from a developers view, but maybe the users are more familiar with something like downloadPlugin and uploadPlugin?

          Show
          mbrohl Michael Brohl added a comment - That would be ok from a developers view, but maybe the users are more familiar with something like downloadPlugin and uploadPlugin?
          Hide
          taher Taher Alkhateeb added a comment -

          Hi Michael,

          I thought about using these words but I think it might be misleading. the pullPlugin is downloading + activating + installing + doing the same for dependencies. The pushPlugin is packaging into POM, publishing and uploading.

          So although the words are more familiar I fear people might misinterpret it and think if they call downloadPlugin then it would just download it when it is a very complex operation.

          This is why I suggest to keep them as pull and push but provide a good explanation in the description, because you'll know that a pull is not just a download so you'll be careful.

          Show
          taher Taher Alkhateeb added a comment - Hi Michael, I thought about using these words but I think it might be misleading. the pullPlugin is downloading + activating + installing + doing the same for dependencies. The pushPlugin is packaging into POM, publishing and uploading. So although the words are more familiar I fear people might misinterpret it and think if they call downloadPlugin then it would just download it when it is a very complex operation. This is why I suggest to keep them as pull and push but provide a good explanation in the description, because you'll know that a pull is not just a download so you'll be careful.
          Hide
          taher Taher Alkhateeb added a comment -

          First commit in r1760917 with minor modifications and fixing of typos.

          Show
          taher Taher Alkhateeb added a comment - First commit in r1760917 with minor modifications and fixing of typos.
          Hide
          taher Taher Alkhateeb added a comment -

          In reference to the discussion thread in which we voted to rename /specialpurpose to /plugins found here I have committed the change in r1778271

          Show
          taher Taher Alkhateeb added a comment - In reference to the discussion thread in which we voted to rename /specialpurpose to /plugins found here I have committed the change in r1778271
          Hide
          taher Taher Alkhateeb added a comment -

          The plugin system is now (more-or-less) complete, with the last step being the renaming of /specialpurpose to /plugins. Thus I'm closing this JIRA and future work on modifying the plugin system would go elsewhere.

          Show
          taher Taher Alkhateeb added a comment - The plugin system is now (more-or-less) complete, with the last step being the renaming of /specialpurpose to /plugins. Thus I'm closing this JIRA and future work on modifying the plugin system would go elsewhere.

            People

            • Assignee:
              taher Taher Alkhateeb
              Reporter:
              taher Taher Alkhateeb
            • Votes:
              3 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development