Uploaded image for project: 'OFBiz'
  1. OFBiz
  2. OFBIZ-7888 Apache OFBiz and OPS
  3. OFBIZ-7910

Improve component template(s) to work with the Gradle solution

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Trivial
    • Resolution: Implemented
    • Affects Version/s: None
    • Fix Version/s: 16.11.01
    • Component/s: framework
    • Labels:
      None

      Description

      Currently the component generated with the createComponent task doesn't have the means to work with the Gradle build solution visavis dependency management.

      The reason for this is that no build.gradle template exists that need to be copied into the new component.

      1. ADD_LIB.patch
        1 kB
        Taher Alkhateeb
      2. OFBIZ-7910-Resources-build.gradle.patch
        1 kB
        Pierre Smits

        Issue Links

          Activity

          Hide
          pfm.smits Pierre Smits added a comment -

          This patch addresses the issue.

          Show
          pfm.smits Pierre Smits added a comment - This patch addresses the issue.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          This is interesting Pierre, but I'd wait to have more directions and possibly actions to put in this file. When looking at Taher's last explanations about the differences between hot-deploy and "plugins" components http://markmail.org/message/w57kadgs5kewwvjs

          * The difference between createComponent and createPlugin is that the plugin resides in /plugins instead of hot-deploy and added to component-load.xml and also contains a build.gradle file designed specifically for plugins. This script contains standard tasks like install, uninstall, perhaps even upgrade. The magic work for plugins will be in their build scripts to integrate tightly with OFBiz.

          I begin to wonder if hot-deploy components would not be less interesting than "plugins" components. Actually, I think we need to think more about the differences between hot-deploy and "plugins" components, and maybe dare to ask this question: "Could we not have only plugins?". Will it else not be confusing for users? Maybe another discussion about those 2 concepts is worth a new thread on dev ML?

          Show
          jacques.le.roux Jacques Le Roux added a comment - This is interesting Pierre, but I'd wait to have more directions and possibly actions to put in this file. When looking at Taher's last explanations about the differences between hot-deploy and "plugins" components http://markmail.org/message/w57kadgs5kewwvjs * The difference between createComponent and createPlugin is that the plugin resides in /plugins instead of hot-deploy and added to component-load.xml and also contains a build.gradle file designed specifically for plugins. This script contains standard tasks like install, uninstall, perhaps even upgrade. The magic work for plugins will be in their build scripts to integrate tightly with OFBiz. I begin to wonder if hot-deploy components would not be less interesting than "plugins" components. Actually, I think we need to think more about the differences between hot-deploy and "plugins" components, and maybe dare to ask this question: "Could we not have only plugins?". Will it else not be confusing for users? Maybe another discussion about those 2 concepts is worth a new thread on dev ML?
          Hide
          taher Taher Alkhateeb added a comment -

          I think we can close this JIRA as not an issue.

          The build.gradle in any component is optional. We should not introduce build.gradle unless there is a specific reason. The gradle system will consider every component (whether it has a build.gradle or not) as a subproject as long as it is an active component. The definition of an active component is:

          • It exists as a folder / directory
          • It exists in component-load.xml if it resides in /applications, /specialpurpose or /framework
          • Or it exists in /hot-deploy (all components are active by default in here).
          Show
          taher Taher Alkhateeb added a comment - I think we can close this JIRA as not an issue. The build.gradle in any component is optional. We should not introduce build.gradle unless there is a specific reason. The gradle system will consider every component (whether it has a build.gradle or not) as a subproject as long as it is an active component. The definition of an active component is: It exists as a folder / directory It exists in component-load.xml if it resides in /applications, /specialpurpose or /framework Or it exists in /hot-deploy (all components are active by default in here).
          Hide
          taher Taher Alkhateeb added a comment -

          You actually make an interesting point Jacques. If we indeed introduce the plugin APIs that I described in the email thread you mentioned above, then it would make the hot-deploy not very necessary. So the user instead of calling createComponent would instead call createPlugin and activatePlugin and the rest of the APIs to control its behavior.

          Nice observation! you flicked a light bulb in my head.

          Show
          taher Taher Alkhateeb added a comment - You actually make an interesting point Jacques. If we indeed introduce the plugin APIs that I described in the email thread you mentioned above, then it would make the hot-deploy not very necessary. So the user instead of calling createComponent would instead call createPlugin and activatePlugin and the rest of the APIs to control its behavior. Nice observation! you flicked a light bulb in my head.
          Hide
          pfm.smits Pierre Smits added a comment -

          I suggest you discuss this in the existing thread in the dev ml.

          Show
          pfm.smits Pierre Smits added a comment - I suggest you discuss this in the existing thread in the dev ml.
          Hide
          pfm.smits Pierre Smits added a comment -

          Hi Taher Alkhateeb,

          It seems you are jumping the gun.. The discusion whether the project will have a plugin management solutions is still ongoing in the dev ml. Before there is an acceptable proof of concept, there will be a lot of water under the bridge.

          Furthermore, as already explained before and elsewhere, the createComponent generates a skeleton of a standard component, that enables a jumpstart for developers. If that new component needs specific external libraries, it will need a build.gradle file in the component. Similar as we have in current specialpurpose component lucene. And that is what the patch provides as part of that skeleton, as a convenience to the developer.

          Show
          pfm.smits Pierre Smits added a comment - Hi Taher Alkhateeb , It seems you are jumping the gun.. The discusion whether the project will have a plugin management solutions is still ongoing in the dev ml. Before there is an acceptable proof of concept, there will be a lot of water under the bridge. Furthermore, as already explained before and elsewhere, the createComponent generates a skeleton of a standard component, that enables a jumpstart for developers. If that new component needs specific external libraries, it will need a build.gradle file in the component. Similar as we have in current specialpurpose component lucene. And that is what the patch provides as part of that skeleton, as a convenience to the developer.
          Hide
          taher Taher Alkhateeb added a comment -

          Noted, in this case I would consider the attached patch as incomplete. It only includes the compile time dependencies format.

          Show
          taher Taher Alkhateeb added a comment - Noted, in this case I would consider the attached patch as incomplete. It only includes the compile time dependencies format.
          Hide
          pfm.smits Pierre Smits added a comment -

          I wonder what it is that you consider to be missing in the patch. It is based on the build.gradle file in the lucene component. That contains nothing else than just the construct for downloading the required external libraries.

          Show
          pfm.smits Pierre Smits added a comment - I wonder what it is that you consider to be missing in the patch. It is based on the build.gradle file in the lucene component. That contains nothing else than just the construct for downloading the required external libraries.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Should we not close this issue and create one for the plugins idea? Else we could simply rename it, what about "Plugins instead of specialpurpose and hot-deploy"?

          Show
          jacques.le.roux Jacques Le Roux added a comment - Should we not close this issue and create one for the plugins idea? Else we could simply rename it, what about "Plugins instead of specialpurpose and hot-deploy"?
          Hide
          pfm.smits Pierre Smits added a comment -

          That is an idea potentially to follow. But to date there is no accepted new feature (regarding the plugin proposal). Having the patch in doesn't bite that process. If and when an acceptable proposal has been implemented anything superseded can be put to the attic, right?

          Show
          pfm.smits Pierre Smits added a comment - That is an idea potentially to follow. But to date there is no accepted new feature (regarding the plugin proposal). Having the patch in doesn't bite that process. If and when an acceptable proposal has been implemented anything superseded can be put to the attic, right?
          Hide
          taher Taher Alkhateeb added a comment -

          I see no reason to waste efforts on duplicate work with the limited resources we have. The plug-in solution seems to be having appeal I think so better to save our energy to more useful and comprehensive solutions instead of creating something to throw it away eventually.

          Show
          taher Taher Alkhateeb added a comment - I see no reason to waste efforts on duplicate work with the limited resources we have. The plug-in solution seems to be having appeal I think so better to save our energy to more useful and comprehensive solutions instead of creating something to throw it away eventually.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          I change the status to trivial for now, and wonder if we should not rather close this issue and document that in wiki, maybe both?

          But as Taher said better to focus our scarce spare time and energy on discussing a plugins solution on the dev ML!

          BTW about documentation, I have created the https://cwiki.apache.org/confluence/display/OFBIZ/From+Ant+to+Gradle but because I have to read a lot of other things related with Gradle did not even find the time to think about it :/

          Show
          jacques.le.roux Jacques Le Roux added a comment - I change the status to trivial for now, and wonder if we should not rather close this issue and document that in wiki, maybe both? But as Taher said better to focus our scarce spare time and energy on discussing a plugins solution on the dev ML! BTW about documentation, I have created the https://cwiki.apache.org/confluence/display/OFBIZ/From+Ant+to+Gradle but because I have to read a lot of other things related with Gradle did not even find the time to think about it :/
          Hide
          taher Taher Alkhateeb added a comment -

          The nice thing about Gradle is that it considers Ant a first class citizen and inter-operates with it very nicely. So for all practical reasons, you can refer to the guide about using ant from gradle (link below). One of the reasons we were able to migrate to Gradle is that we used Ant in the master build script to apply functionality that used to exist in the old ant scripts (like the junitreport). You can also put a few code snippets in that document that shows how to import an ant file. The master build.gradle file itself has a few nice references on how to use ant from gradle.

          So I would say if you did not make any customizations to OFBiz, then you don't need to do anything to migrate. If, on the other hand, you did make custom ant scripts let's say, then you just need to import them into your build.gradle file (whether top level or inside components).

          In short, you can just write a paragraph explaining that gradle works with ant, provide the below link, and maybe a little code snippet to show how to import ant scripts. I think this would be satisfactory to users unless I'm missing something important?

          https://docs.gradle.org/current/userguide/ant.html

          Show
          taher Taher Alkhateeb added a comment - The nice thing about Gradle is that it considers Ant a first class citizen and inter-operates with it very nicely. So for all practical reasons, you can refer to the guide about using ant from gradle (link below). One of the reasons we were able to migrate to Gradle is that we used Ant in the master build script to apply functionality that used to exist in the old ant scripts (like the junitreport). You can also put a few code snippets in that document that shows how to import an ant file. The master build.gradle file itself has a few nice references on how to use ant from gradle. So I would say if you did not make any customizations to OFBiz, then you don't need to do anything to migrate. If, on the other hand, you did make custom ant scripts let's say, then you just need to import them into your build.gradle file (whether top level or inside components). In short, you can just write a paragraph explaining that gradle works with ant, provide the below link, and maybe a little code snippet to show how to import ant scripts. I think this would be satisfactory to users unless I'm missing something important? https://docs.gradle.org/current/userguide/ant.html
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Thanks Taher, I did not get the time to read much of the excellent Gradle documentation, having to follow and answer to a lot of related Jira issues. I believe we are a bit sidetracked, with indeed issue like this one!

          I think, after a current global review on existing Gradle Jira issues, I'll more focus on documentation (both read and write) and my "own" Gradle issues

          Show
          jacques.le.roux Jacques Le Roux added a comment - Thanks Taher, I did not get the time to read much of the excellent Gradle documentation, having to follow and answer to a lot of related Jira issues. I believe we are a bit sidetracked, with indeed issue like this one! I think, after a current global review on existing Gradle Jira issues, I'll more focus on documentation (both read and write) and my "own" Gradle issues
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Pierre from your last comment at OFBIZ-7948, and more as a reminder for our documentation, we should note that when using external libs in (hopefully soon) plugins, the local build.gradle file must also contain

          apply plugin:'java'

          before the dependencies definition.

          Show
          jacques.le.roux Jacques Le Roux added a comment - Pierre from your last comment at OFBIZ-7948 , and more as a reminder for our documentation, we should note that when using external libs in (hopefully soon) plugins, the local build.gradle file must also contain apply plugin:'java' before the dependencies definition.
          Hide
          taher Taher Alkhateeb added a comment -

          Hi Jacques,

          You can apply the java plugin directly to subprojects from the master build script. But why? We don't need the Java plugin in subprojects, that would just create an unnecessary build directory with artifacts that we will not use.

          Show
          taher Taher Alkhateeb added a comment - Hi Jacques, You can apply the java plugin directly to subprojects from the master build script. But why? We don't need the Java plugin in subprojects, that would just create an unnecessary build directory with artifacts that we will not use.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Hi Taher,

          So what would be your answer to Wai at OFBIZ-7948 ?

          Show
          jacques.le.roux Jacques Le Roux added a comment - Hi Taher, So what would be your answer to Wai at OFBIZ-7948 ?
          Hide
          taher Taher Alkhateeb added a comment -

          I'm not sure if I got it right, but by default gradle will look in every component for a /lib directory and add any of those jars to the classpath. Specifically, this piece of code already does the job automatically as long as you put those jars in /lib

          getDirectoryInActiveComponentsIfExists('lib').each { libDir ->
              compile fileTree(dir: libDir, include: '**/*.jar')
          }
          
          Show
          taher Taher Alkhateeb added a comment - I'm not sure if I got it right, but by default gradle will look in every component for a /lib directory and add any of those jars to the classpath. Specifically, this piece of code already does the job automatically as long as you put those jars in /lib getDirectoryInActiveComponentsIfExists('lib').each { libDir -> compile fileTree(dir: libDir, include: '**/*.jar') }
          Hide
          wt Wai added a comment -

          Hello Taher, Jacques

          Here are my findings.

          My source tree contains the following relevant details...
          /ofbiz
          /ofbiz/lib
          /ofbiz/hot-deploy/mycomponent1
          /ofbiz/hot-deploy/mycomponent1/lib

          mycomponent1 references a class in myInHouse.jar

          Try1:
          I placed the myInHouse.jar in /ofbiz/lib
          Modified /ofbiz/build.gradle and added the following to dependencies
          compile files('lib')
          Ran $gradlew cleanAll build
          The compilation failed on mycomponent1.
          -the failure is due to mycomponent1 referencing a class that is not found. This class is in myInHouse.jar

          Try2:
          I placed the myInHouse.jar into /ofbiz/hot-deploy/mycomponent1/lib
          Ran $gradlew cleanAll build
          The compilation was sucessful.

          Conclusion:
          My findinds corroborates Taher's comment about searching in the local 'lib' directory of each component to locate the custom jars.
          I also noticed a deficiency with the build script. Consider a developer creates multiple hot-deploy components. Each component references classes from the same myInHouse.jar. This would require a copy of the same jar in each of the components 'lib' directory. This is very inconvenient.
          I suggest to modify the script so that each hot-deploy component can reference one myInHouse.jar located in /ofbiz/lib.

          I hope this helps.
          Wai

          Show
          wt Wai added a comment - Hello Taher, Jacques Here are my findings. My source tree contains the following relevant details... /ofbiz /ofbiz/lib /ofbiz/hot-deploy/mycomponent1 /ofbiz/hot-deploy/mycomponent1/lib mycomponent1 references a class in myInHouse.jar Try1: I placed the myInHouse.jar in /ofbiz/lib Modified /ofbiz/build.gradle and added the following to dependencies compile files('lib') Ran $gradlew cleanAll build The compilation failed on mycomponent1. -the failure is due to mycomponent1 referencing a class that is not found. This class is in myInHouse.jar Try2: I placed the myInHouse.jar into /ofbiz/hot-deploy/mycomponent1/lib Ran $gradlew cleanAll build The compilation was sucessful. Conclusion: My findinds corroborates Taher's comment about searching in the local 'lib' directory of each component to locate the custom jars. I also noticed a deficiency with the build script. Consider a developer creates multiple hot-deploy components. Each component references classes from the same myInHouse.jar. This would require a copy of the same jar in each of the components 'lib' directory. This is very inconvenient. I suggest to modify the script so that each hot-deploy component can reference one myInHouse.jar located in /ofbiz/lib. I hope this helps. Wai
          Hide
          taher Taher Alkhateeb added a comment -

          Hi Wai,

          Any library you import is shared by all components under the new gradle build system. The problem in your modification in "Try1" is that the syntax is not right. Please take a look at the above code for proper importing of libs. If you import properly then your components would not complain. You just need to add $OFBIZ_HOME/lib to the compile set.

          Show
          taher Taher Alkhateeb added a comment - Hi Wai, Any library you import is shared by all components under the new gradle build system. The problem in your modification in "Try1" is that the syntax is not right. Please take a look at the above code for proper importing of libs. If you import properly then your components would not complain. You just need to add $OFBIZ_HOME/lib to the compile set.
          Hide
          wt Wai added a comment -

          I followed your suggestion by modifying /ofbiz/build.gradle and replace the dependency to
          compile files('$OFBIZ_HOME/lib') and placing myInHouse.jar into /ofbiz/lib
          After running $ gradlew cleanAll build, a compilation error occurs with mycomponent1 with symbol not found.

          Show
          wt Wai added a comment - I followed your suggestion by modifying /ofbiz/build.gradle and replace the dependency to compile files('$OFBIZ_HOME/lib') and placing myInHouse.jar into /ofbiz/lib After running $ gradlew cleanAll build, a compilation error occurs with mycomponent1 with symbol not found.
          Hide
          taher Taher Alkhateeb added a comment -

          Hi Wai, i'm not sure if you meant this literally or what, so let me try to be very exact:

          If you want to include your common libs in root folder /lib use the below code:

          compile fileTree(dir: file("${rootDir}/lib"), include: '**/*.jar')
          

          Alternatively (and more simply) just put your shared libs in /framework/base/lib/

          Show
          taher Taher Alkhateeb added a comment - Hi Wai, i'm not sure if you meant this literally or what, so let me try to be very exact: If you want to include your common libs in root folder /lib use the below code: compile fileTree(dir: file( "${rootDir}/lib" ), include: '**/*.jar') Alternatively (and more simply) just put your shared libs in /framework/base/lib/
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Taher,

          I was under the impression that after the jpim-0.1.jar removing we would get rid of /framework/base/lib/. Do you have other plans in mind? Is your answer only a temporay solution?

          Show
          jacques.le.roux Jacques Le Roux added a comment - Taher, I was under the impression that after the jpim-0.1.jar removing we would get rid of /framework/base/lib/. Do you have other plans in mind? Is your answer only a temporay solution?
          Hide
          taher Taher Alkhateeb added a comment -

          You are absolutely right and you guessed right, my answer is temporary. Right now I'm just trying to help Wai to make sure there are no bugs in the build system (I believe there are none as far as lib importing is concerned).

          Perhaps a permanent solution is to get rid of all /lib folders and just keep an empty one in root folder. Given that we do not really need the custom classloader and we can just share the libraries with all components (cleaner, easier, integrated and detects conflicts early).

          So if we do decide to adopt the permanent solution described above, we can use the code snippet I provided to Wai. No strong opinion, whatever you guys prefer.

          Show
          taher Taher Alkhateeb added a comment - You are absolutely right and you guessed right, my answer is temporary. Right now I'm just trying to help Wai to make sure there are no bugs in the build system (I believe there are none as far as lib importing is concerned). Perhaps a permanent solution is to get rid of all /lib folders and just keep an empty one in root folder. Given that we do not really need the custom classloader and we can just share the libraries with all components (cleaner, easier, integrated and detects conflicts early). So if we do decide to adopt the permanent solution described above, we can use the code snippet I provided to Wai. No strong opinion, whatever you guys prefer.
          Hide
          wt Wai added a comment -

          Thanks, that worked.
          Wai

          Show
          wt Wai added a comment - Thanks, that worked. Wai
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Wai, could you please confirm which way you used? Because we all want a definitive solution... Thanks!

          Show
          jacques.le.roux Jacques Le Roux added a comment - Wai, could you please confirm which way you used? Because we all want a definitive solution... Thanks!
          Hide
          wt Wai added a comment - - edited

          In /ofbiz/build.gradle, I added Taher's suggestion in the 'dependencies' block

          dependencies {
              ...
              compile fileTree(dir: file("${rootDir}/lib"), include: '**/*.jar')
              ...
          }
          

          Then I added the 3rd party in-house jar into /ofbiz/lib.
          Ran $ gradle cleanAll build
          Now my hot-deploy component compiles ok.

          Show
          wt Wai added a comment - - edited In /ofbiz/build.gradle, I added Taher's suggestion in the 'dependencies' block dependencies { ... compile fileTree(dir: file( "${rootDir}/lib" ), include: '**/*.jar') ... } Then I added the 3rd party in-house jar into /ofbiz/lib. Ran $ gradle cleanAll build Now my hot-deploy component compiles ok.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Thanks Wai, it's clear now

          Show
          jacques.le.roux Jacques Le Roux added a comment - Thanks Wai, it's clear now
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          It seems the best way to me, no needs to dig down the folders tree! And as you said it's

          cleaner, easier, integrated and detects conflicts early

          Better indeed to have only one version by lib, notably for security reasons!

          Show
          jacques.le.roux Jacques Le Roux added a comment - It seems the best way to me, no needs to dig down the folders tree! And as you said it's cleaner, easier, integrated and detects conflicts early Better indeed to have only one version by lib, notably for security reasons!
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Taher, I let you implement your idea OOTB, thanks!

          Show
          jacques.le.roux Jacques Le Roux added a comment - Taher, I let you implement your idea OOTB, thanks!
          Hide
          taher Taher Alkhateeb added a comment -

          Hello, I am attaching a patch that creates a custom libs folder in OFBiz top level directory where people can keep their jars and update the master script build.gradle accordingly. Take a look at it and see if suitable.

          Show
          taher Taher Alkhateeb added a comment - Hello, I am attaching a patch that creates a custom libs folder in OFBiz top level directory where people can keep their jars and update the master script build.gradle accordingly. Take a look at it and see if suitable.
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Thanks Taher, this seems good to me and we will only need to use grunt-md2html to put (and later possibly update) the documentation in the wiki. If nobody disagree I'll commit in few days

          Show
          jacques.le.roux Jacques Le Roux added a comment - Thanks Taher, this seems good to me and we will only need to use grunt-md2html to put (and later possibly update) the documentation in the wiki. If nobody disagree I'll commit in few days
          Hide
          jacques.le.roux Jacques Le Roux added a comment -

          Thanks Taher, Pierre, Wai,

          Taher your patch is in trunk at revision: 1756515

          I simply "svn:ignored" the libs folder since only specific users libs will get there

          Show
          jacques.le.roux Jacques Le Roux added a comment - Thanks Taher, Pierre, Wai, Taher your patch is in trunk at revision: 1756515 I simply "svn:ignored" the libs folder since only specific users libs will get there

            People

            • Assignee:
              jacques.le.roux Jacques Le Roux
              Reporter:
              pfm.smits Pierre Smits
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development