Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Commons Parent Pom
    • Labels:
      None

      Description

      Opening this ticket to discuss changes for Version 6 of the commons-parent pom.

      See thread: http://tinyurl.com/39eo9z for related discussion/issues

      1. commons-valdiator-generated-NOTICE.txt
        0.7 kB
        Niall Pemberton
      2. COMMONSSITE-21-commons-parent-pom-v2.patch
        2 kB
        Niall Pemberton
      3. COMMONSSITE-21-commons-parent-pom-v1.patch
        3 kB
        Niall Pemberton

        Activity

        Niall Pemberton created issue -
        Hide
        Niall Pemberton added a comment -

        Attaching a patch for the commons-parent pom which does the following:

        1) Configure the maven-release-plugin in the "rc" profile with an "arguments" value of "-Prc"
        2) Configure the maven-release-plugin in the "release" profile with an "arguments" value of "-Prelease"

        Trying to release commons-skin I found that the deploy step was not picking up the the repository from the specified profile. So for example using:
        mvn -Prc perform:release
        ignored the configured repository for the "rc" profile and tried to deploy to the "dummy" repository specified in commons-parent pom.

        Without actually releasing something I can't test whether this proposed solution will work - but the commons logging pom has a similar configuration for that plugin (but in the build profile)
        http://svn.apache.org/repos/asf/commons/proper/logging/trunk/pom.xml

        3) Remove the maven-remote-resources-plugin (from "rc" and "release" profiles)
        Currently this plugin is automatically adding "License" and "Notice" files (without the .txt extension) to the meta-inf directory in jars - since all our components already have NOTICE.txt and LICENSE.txt files which are configured as "resources" this causes duplicate license/notice files to be added to the jar. Also the generated notice file adds entries for dependencies - which if they don't have appropriate organization/license info sepcified in their poms says "developed by an
        unknown organization" - this doesn't look too good, especially when the dependencies are Apache projects (examples BeanUtils 1.7.0 and Jakarta ORO)

        Show
        Niall Pemberton added a comment - Attaching a patch for the commons-parent pom which does the following: 1) Configure the maven-release-plugin in the "rc" profile with an "arguments" value of "-Prc" 2) Configure the maven-release-plugin in the "release" profile with an "arguments" value of "-Prelease" Trying to release commons-skin I found that the deploy step was not picking up the the repository from the specified profile. So for example using: mvn -Prc perform:release ignored the configured repository for the "rc" profile and tried to deploy to the "dummy" repository specified in commons-parent pom. Without actually releasing something I can't test whether this proposed solution will work - but the commons logging pom has a similar configuration for that plugin (but in the build profile) http://svn.apache.org/repos/asf/commons/proper/logging/trunk/pom.xml 3) Remove the maven-remote-resources-plugin (from "rc" and "release" profiles) Currently this plugin is automatically adding "License" and "Notice" files (without the .txt extension) to the meta-inf directory in jars - since all our components already have NOTICE.txt and LICENSE.txt files which are configured as "resources" this causes duplicate license/notice files to be added to the jar. Also the generated notice file adds entries for dependencies - which if they don't have appropriate organization/license info sepcified in their poms says "developed by an unknown organization" - this doesn't look too good, especially when the dependencies are Apache projects (examples BeanUtils 1.7.0 and Jakarta ORO)
        Niall Pemberton made changes -
        Field Original Value New Value
        Attachment COMMONSSITE-21-commons-parent-pom-v1.patch [ 12371402 ]
        Niall Pemberton made changes -
        Component/s Commons Parent [ 12312074 ]
        Hide
        Rahul Akolkar added a comment -

        Changes look OK to me (haven't tried them either) – I also think the NOTICEs generated by the remote resources plugin are not yet usable.

        Show
        Rahul Akolkar added a comment - Changes look OK to me (haven't tried them either) – I also think the NOTICEs generated by the remote resources plugin are not yet usable.
        Hide
        Dennis Lundberg added a comment -

        1 and 2 sound fine to me. Specifying the version of maven-release-plugin is definitely needed.

        Regarding 3 I disagree. I think we should do it the other way around. If we are going to have Maven 2 as the default build environment in Commons we should use standard Maven configuration in commons-parent. When a component, for whatever reason, needs to deviate from the standard way, we should add fixes or workarounds in that component. Using maven-remote-resources-plugin is the way to go within the ASF for projects using Maven. If the data it uses (that goes into the NOTICE file) isn't to our liking, then let's help fix the data.

        Show
        Dennis Lundberg added a comment - 1 and 2 sound fine to me. Specifying the version of maven-release-plugin is definitely needed. Regarding 3 I disagree. I think we should do it the other way around. If we are going to have Maven 2 as the default build environment in Commons we should use standard Maven configuration in commons-parent. When a component, for whatever reason, needs to deviate from the standard way, we should add fixes or workarounds in that component. Using maven-remote-resources-plugin is the way to go within the ASF for projects using Maven. If the data it uses (that goes into the NOTICE file) isn't to our liking, then let's help fix the data.
        Hide
        Rahul Akolkar added a comment -

        Its a bootstrapping problem when asking for adoption with sub-optimal underlying data. There is hardly a doubt this is useful when it works, but if we have to workaround it for the majority of components then it starts to get cumbersome to have it in the parent.

        Another way to look at this is doing it ground up, where when it is useful for a majority of components, we move it to the parent pom.

        Show
        Rahul Akolkar added a comment - Its a bootstrapping problem when asking for adoption with sub-optimal underlying data. There is hardly a doubt this is useful when it works, but if we have to workaround it for the majority of components then it starts to get cumbersome to have it in the parent. Another way to look at this is doing it ground up, where when it is useful for a majority of components, we move it to the parent pom.
        Hide
        Dennis Lundberg added a comment -

        I have studied the current svn trunk version of project.xml and pom.xml files for all commons projects. For multiprojects like jelly and jci I just looked at the component parent.

        All project.xml files include an <organization> element. All pom.xml files inherit from commons-parent-5, which inherits from apache-parent-4, which includes an <organization> element. So any commons release from here on will have a proper <organization> element.

        Looking at dependencies on other commons components in all current svn trunk project.xml files I found that the following components lacks an <organization> element in their poms in the central repository. Each component is a dependency in only one other component:

        • pool-1.1
        • discovery-0.2
        • digester-1.4.1
        • codec-1.2

        Then we have beanutils-1.7, which is a dependency in many other common components. Something has gone wrong when this was deployed to the repository. The M1 repo is missing the pom all together and the M2 repo has a pom that cannot stem from the original project.xml in the svn tag. This is something that can probably fixed in the repository because it is not the correct pom that has been deployed. If no one objects I will open a JIRA for this over at http://jira.codehaus.org/browse/MEV to see if we can get it fixed.

        As you can see there are very few of our own dependencies that have bad meta data in the central repository, with beanutils-1.7 being the exception to the rule here. So I believe that using maven-remote-resources-plugin will work for most commons components when it is time to release its next version.

        Show
        Dennis Lundberg added a comment - I have studied the current svn trunk version of project.xml and pom.xml files for all commons projects. For multiprojects like jelly and jci I just looked at the component parent. All project.xml files include an <organization> element. All pom.xml files inherit from commons-parent-5, which inherits from apache-parent-4, which includes an <organization> element. So any commons release from here on will have a proper <organization> element. Looking at dependencies on other commons components in all current svn trunk project.xml files I found that the following components lacks an <organization> element in their poms in the central repository. Each component is a dependency in only one other component: pool-1.1 discovery-0.2 digester-1.4.1 codec-1.2 Then we have beanutils-1.7, which is a dependency in many other common components. Something has gone wrong when this was deployed to the repository. The M1 repo is missing the pom all together and the M2 repo has a pom that cannot stem from the original project.xml in the svn tag. This is something that can probably fixed in the repository because it is not the correct pom that has been deployed. If no one objects I will open a JIRA for this over at http://jira.codehaus.org/browse/MEV to see if we can get it fixed. As you can see there are very few of our own dependencies that have bad meta data in the central repository, with beanutils-1.7 being the exception to the rule here. So I believe that using maven-remote-resources-plugin will work for most commons components when it is time to release its next version.
        Hide
        Rahul Akolkar added a comment -

        ORO was mentioned, theres also xalan and xml-apis that I ran into the other day, as examples of bits that we don't release out of Commons (so its not just the <organization>s in our trunks). Obviously, components that have more dependencies are likely to be bitten by this.

        Show
        Rahul Akolkar added a comment - ORO was mentioned, theres also xalan and xml-apis that I ran into the other day, as examples of bits that we don't release out of Commons (so its not just the <organization>s in our trunks). Obviously, components that have more dependencies are likely to be bitten by this.
        Hide
        Niall Pemberton added a comment -

        Dennis,

        There is a bigger problem with the BeanUtils 1.7.0 poms than the organization (in the m2 repo):

        • [m2] commons-beanutils-1.7.0.pom[1] doesn't specify a dependency (optional) on Commons Collections (which it does have)
        • [m2] commons-beanutils-core-1.7.0.pom[2] specifies a dependency on Commons Collections (which it doesn't have)

        As you said the m1 repo doesn't have a commons-beanutils-1.7.0.pom, but does have a commons-beanutils-core-1.7.0.pom[3] with the same issue

        [1] http://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.pom
        [2] http://repo1.maven.org/maven2/commons-beanutils/commons-beanutils-core/1.7.0/commons-beanutils-core-1.7.0.pom
        [3] http://repo1.maven.org/maven/commons-beanutils/poms/commons-beanutils-core-1.7.0.pom

        Show
        Niall Pemberton added a comment - Dennis, There is a bigger problem with the BeanUtils 1.7.0 poms than the organization (in the m2 repo): [m2] commons-beanutils-1.7.0.pom [1] doesn't specify a dependency (optional) on Commons Collections (which it does have) [m2] commons-beanutils-core-1.7.0.pom [2] specifies a dependency on Commons Collections (which it doesn't have) As you said the m1 repo doesn't have a commons-beanutils-1.7.0.pom, but does have a commons-beanutils-core-1.7.0.pom [3] with the same issue [1] http://repo1.maven.org/maven2/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.pom [2] http://repo1.maven.org/maven2/commons-beanutils/commons-beanutils-core/1.7.0/commons-beanutils-core-1.7.0.pom [3] http://repo1.maven.org/maven/commons-beanutils/poms/commons-beanutils-core-1.7.0.pom
        Hide
        Niall Pemberton added a comment -

        The ideal solution IMO to the remote-resources-plugin issues (duplicate LICENSE/NOTICE entries and the quality of the generated NOTICE file) would be that if the project is already configured to add LICENSE/NOTICE files to the jar(2) then the remote-resources-plugin doesn't then add duplicate copies (Note: it needs to be smart since all our projects use LICENSE.txt/NOTICE.txt and it adds files named License/Notice).

        Show
        Niall Pemberton added a comment - The ideal solution IMO to the remote-resources-plugin issues (duplicate LICENSE/NOTICE entries and the quality of the generated NOTICE file) would be that if the project is already configured to add LICENSE/NOTICE files to the jar(2) then the remote-resources-plugin doesn't then add duplicate copies (Note: it needs to be smart since all our projects use LICENSE.txt/NOTICE.txt and it adds files named License/Notice).
        Niall Pemberton committed 603888 (1 file)
        Reviews: none

        COMMONSSITE-21 - Configure the maven-release-plugin in the "rc" and "release" profiles with an "arguments" to specify the profile. This will (hopefully) resolve the issue of the deploy plugin not picking up the distribution management from the correct profile.

        Hide
        Niall Pemberton added a comment -

        OK I've committed 1) and 2) since nobody disagreed with those changes:

        http://svn.apache.org/viewvc?view=rev&revision=603888

        Show
        Niall Pemberton added a comment - OK I've committed 1) and 2) since nobody disagreed with those changes: http://svn.apache.org/viewvc?view=rev&revision=603888
        Hide
        Niall Pemberton added a comment -

        Attaching v2 of the patch as v1 has been partially committed (just contains the remote resources plugin changes).

        Show
        Niall Pemberton added a comment - Attaching v2 of the patch as v1 has been partially committed (just contains the remote resources plugin changes).
        Niall Pemberton made changes -
        Hide
        Niall Pemberton added a comment -

        Since people are unaware of what the remote resources plugin actually does wrt to NOTICE files I'm attaching an example NOTICE file that was generated for Commons Validator by that plugin

        Show
        Niall Pemberton added a comment - Since people are unaware of what the remote resources plugin actually does wrt to NOTICE files I'm attaching an example NOTICE file that was generated for Commons Validator by that plugin
        Niall Pemberton made changes -
        Attachment commons-valdiator-generated-NOTICE.txt [ 12371668 ]
        Hide
        Simon Kitching added a comment -

        I'm not sure this works right for NOTICE files.

        For example, commons-logging might include some code written by a third party but licensed under an APL2.0-compatible license (including BSD, etc). In that case AIUI we remove copyright and license information from the files [1], and put it into NOTICE instead. This keeps the code uncluttered, while putting the contributor info for the project ancestry into one easy-to-find place (NOTICE.txt).

        So it is not just a matter of taking the organisation property from the pom; a single mvn module can have multiple entries in its NOTICE file.

        To be doubly-clear: a single maven module (that is, a single pom.xml file) may correspond to many lines of text in the NOTICE file, acknowledging every single copyright holder to code contained in that single maven module. AFAIK, it is not possible or appropriate to put one organization tag in a pom for every line we need in the NOTICE for that module.

        For a notice inside a single JAR, I don't believe it is a legal requirement to acknowledge the copyright holders of jars it depends on. In fact I would prefer to NOT do that. The dependency information is explicitly present in the pom, and each of those dependent jars should contain its own acknowledgements.

        Whether an overall NOTICE is needed for a downloadable bundle that contains multiple jars is debatable. I think not; there is a LICENSE file (APL2 of course) which is all that is relevant for most people.

        Show
        Simon Kitching added a comment - I'm not sure this works right for NOTICE files. For example, commons-logging might include some code written by a third party but licensed under an APL2.0-compatible license (including BSD, etc). In that case AIUI we remove copyright and license information from the files [1] , and put it into NOTICE instead. This keeps the code uncluttered, while putting the contributor info for the project ancestry into one easy-to-find place (NOTICE.txt). So it is not just a matter of taking the organisation property from the pom; a single mvn module can have multiple entries in its NOTICE file. To be doubly-clear: a single maven module (that is, a single pom.xml file) may correspond to many lines of text in the NOTICE file, acknowledging every single copyright holder to code contained in that single maven module. AFAIK, it is not possible or appropriate to put one organization tag in a pom for every line we need in the NOTICE for that module. For a notice inside a single JAR, I don't believe it is a legal requirement to acknowledge the copyright holders of jars it depends on. In fact I would prefer to NOT do that. The dependency information is explicitly present in the pom, and each of those dependent jars should contain its own acknowledgements. Whether an overall NOTICE is needed for a downloadable bundle that contains multiple jars is debatable. I think not; there is a LICENSE file (APL2 of course) which is all that is relevant for most people.
        Hide
        Sebb added a comment -

        The example NOTICE file does not really show how 3rd party software would be handled.

        [There are also some typos in it: software(s) should be software, and jakarta should be removed]

        Also, it's actually the LICENSE file that needs to contain ALL the licenses. (My e-mail said NOTICE, which was wrong)
        Or the LICENSE file can contain the text of the AL + pointers to the other licence files, which need to be in SVN as well.

        I can dig out discussions of this if necessary (went through this with JMeter)

        Show
        Sebb added a comment - The example NOTICE file does not really show how 3rd party software would be handled. [There are also some typos in it: software(s) should be software, and jakarta should be removed] Also, it's actually the LICENSE file that needs to contain ALL the licenses. (My e-mail said NOTICE, which was wrong) Or the LICENSE file can contain the text of the AL + pointers to the other licence files, which need to be in SVN as well. I can dig out discussions of this if necessary (went through this with JMeter)
        Hide
        Niall Pemberton added a comment -

        The "ASF Source Header and Copyright Notice Policy" [1] deals with the contents (example[2] from that doc) of the NOTICE file (is there any other relevant policy docs?) - to be honest I find it a little vague since it uses words like "may" rather than "must".

        I agree though that for anything more complex than the AL2 its probably too blunt an instrument - but do we have any components currently in commons that have the scenarios you are bringing up? Because if we don't then isn't it pointless raising theoretical objections. For me the remote resources plugin isn't quite their yet - but down the road I could see it working well for most components. The best solution IMO is what I suggested earlier - if the remote resources plugin only generated a NOTICE file if there wasn''t already one specified in the projects resources. That way components that needed all sorts of 3rd party stuff could manually craft their NOTICE file - while the majority just get it generated.

        [1] http://www.apache.org/legal/src-headers.html
        [2] http://www.apache.org/licenses/example-NOTICE.txt

        Show
        Niall Pemberton added a comment - The "ASF Source Header and Copyright Notice Policy" [1] deals with the contents (example [2] from that doc) of the NOTICE file (is there any other relevant policy docs?) - to be honest I find it a little vague since it uses words like "may" rather than "must". I agree though that for anything more complex than the AL2 its probably too blunt an instrument - but do we have any components currently in commons that have the scenarios you are bringing up? Because if we don't then isn't it pointless raising theoretical objections. For me the remote resources plugin isn't quite their yet - but down the road I could see it working well for most components. The best solution IMO is what I suggested earlier - if the remote resources plugin only generated a NOTICE file if there wasn''t already one specified in the projects resources. That way components that needed all sorts of 3rd party stuff could manually craft their NOTICE file - while the majority just get it generated. [1] http://www.apache.org/legal/src-headers.html [2] http://www.apache.org/licenses/example-NOTICE.txt
        Hide
        Simon Kitching added a comment -

        Sebb, it would be good if you could find a link to that email thread re LICENSE because that is not my understanding of how it works.

        AIUI, we always distribute software under just the APL2. And that means that anyone who wants to use any part of our software can always do so under the terms of that license. There might be parts that are even more liberal, but it is not our responsibility to point those bits out. So just an APL2 license file is all that is needed.

        I vaguely remember that there are cases where we do distribute the occasional file that has a less liberal licence, eg xml schemas.. But in that case, the file itself carries the necessary license. Of course we only do this where redistribution is legal, and anyone wanting to modify cannot possibly be unaware that the file they are modifying has another license because the terms are in the file. We may need to acknowledge the provider of that software - but that's what NOTICE is for. NOTE: this bit is from memory, don't sue me over it

        Show
        Simon Kitching added a comment - Sebb, it would be good if you could find a link to that email thread re LICENSE because that is not my understanding of how it works. AIUI, we always distribute software under just the APL2. And that means that anyone who wants to use any part of our software can always do so under the terms of that license. There might be parts that are even more liberal, but it is not our responsibility to point those bits out. So just an APL2 license file is all that is needed. I vaguely remember that there are cases where we do distribute the occasional file that has a less liberal licence, eg xml schemas.. But in that case, the file itself carries the necessary license. Of course we only do this where redistribution is legal, and anyone wanting to modify cannot possibly be unaware that the file they are modifying has another license because the terms are in the file. We may need to acknowledge the provider of that software - but that's what NOTICE is for. NOTE: this bit is from memory, don't sue me over it
        Hide
        Simon Kitching added a comment -

        PS: but anyway, I'd rather see an explicit copy of the LICENSE file in SVN in the mvn module root directory than have it pulled in as part of the build. It's much clearer to users, and updating it in the case of a new license being issued by the ASF is not a major concern.

        Show
        Simon Kitching added a comment - PS: but anyway, I'd rather see an explicit copy of the LICENSE file in SVN in the mvn module root directory than have it pulled in as part of the build. It's much clearer to users, and updating it in the case of a new license being issued by the ASF is not a major concern.
        Hide
        Sebb added a comment -

        The contents of the NOTICE file are described here:

        http://www.apache.org/legal/src-headers.html#notice

        and a sample is here:

        http://www.apache.org/licenses/example-NOTICE.txt
        linked from
        http://apache.org/dev/apply-license.html#new

        So the example NOTICE file is not quite correct, as it is supposed to start with the ASF stuff, and must use Apache Commons etc.
        I assume that the // comments at the beginning are not part of the final file; if they are, they should be removed.

        I've not yet found the LICENSE discussion.

        Show
        Sebb added a comment - The contents of the NOTICE file are described here: http://www.apache.org/legal/src-headers.html#notice and a sample is here: http://www.apache.org/licenses/example-NOTICE.txt linked from http://apache.org/dev/apply-license.html#new So the example NOTICE file is not quite correct, as it is supposed to start with the ASF stuff, and must use Apache Commons etc. I assume that the // comments at the beginning are not part of the final file; if they are, they should be removed. I've not yet found the LICENSE discussion.
        Hide
        Sebb added a comment -

        LICENSE file documentation:

        http://www.apache.org/dev/release.html#distributing-code-under-several-licenses

        Example:
        https://svn.apache.org/repos/asf/httpd/httpd/trunk/LICENSE

        I haven't yet found a reference to allowing pointers in the LICENSE file.

        Show
        Sebb added a comment - LICENSE file documentation: http://www.apache.org/dev/release.html#distributing-code-under-several-licenses Example: https://svn.apache.org/repos/asf/httpd/httpd/trunk/LICENSE I haven't yet found a reference to allowing pointers in the LICENSE file.
        Hide
        Niall Pemberton added a comment -

        I had a look through all the NOTICE.txt files in the commons proper (didn't check mulit-module components sub-modules - e.g. attributes, jci, jelly) and all were the standard flavour except for Codec, EL, Math and VFS:

        http://svn.apache.org/repos/asf/commons/proper/codec/trunk/NOTICE.txt
        http://svn.apache.org/repos/asf/commons/proper/el/trunk/NOTICE.txt
        http://svn.apache.org/repos/asf/commons/proper/math/trunk/NOTICE.txt
        http://svn.apache.org/repos/asf/commons/proper/vfs/trunk/NOTICE.txt

        Show
        Niall Pemberton added a comment - I had a look through all the NOTICE.txt files in the commons proper (didn't check mulit-module components sub-modules - e.g. attributes, jci, jelly) and all were the standard flavour except for Codec, EL, Math and VFS: http://svn.apache.org/repos/asf/commons/proper/codec/trunk/NOTICE.txt http://svn.apache.org/repos/asf/commons/proper/el/trunk/NOTICE.txt http://svn.apache.org/repos/asf/commons/proper/math/trunk/NOTICE.txt http://svn.apache.org/repos/asf/commons/proper/vfs/trunk/NOTICE.txt
        Hide
        Sebb added a comment -

        The maths one looks odd - it seems to include licenses, rather than just required attributions.

        The iicenses should probably be in the LICENSE file, as per the example I quoted earier:
        https://svn.apache.org/repos/asf/httpd/httpd/trunk/LICENSE

        Show
        Sebb added a comment - The maths one looks odd - it seems to include licenses, rather than just required attributions. The iicenses should probably be in the LICENSE file, as per the example I quoted earier: https://svn.apache.org/repos/asf/httpd/httpd/trunk/LICENSE
        Hide
        Dennis Lundberg added a comment -

        Sebb,

        Dependencies on 3rd party software will be handled similarly to Apache software in the NOTICE file. If there is a pom available for the 3rd party software, data from that pom will be used in the generated NOTICE file.

        I have corrected the typo "(s)" in the apache-jar-resource-bundle.

        Jakarta should not be removed from the NOTICE file, because the dependencies that Validator has were released by the Jakarta project.

        Regarding the // comments, I think they are OK to have. The docs says "The top of each NOTICE file should include the following text, ...". The NOTICE file includes that text in the top, it's not just the very first thing in the file.

        Getting "Apache ..." on the first row can be configured as I understand it.

        Show
        Dennis Lundberg added a comment - Sebb, Dependencies on 3rd party software will be handled similarly to Apache software in the NOTICE file. If there is a pom available for the 3rd party software, data from that pom will be used in the generated NOTICE file. I have corrected the typo "(s)" in the apache-jar-resource-bundle. Jakarta should not be removed from the NOTICE file, because the dependencies that Validator has were released by the Jakarta project. Regarding the // comments, I think they are OK to have. The docs says "The top of each NOTICE file should include the following text, ...". The NOTICE file includes that text in the top, it's not just the very first thing in the file. Getting "Apache ..." on the first row can be configured as I understand it.
        Hide
        Dennis Lundberg added a comment -

        Niall,

        There is a "skip" parameter in maven-remote-resources-plugin that I think can be used to disable the plugin for components that have needs that cannot be accommodated by the plugin. Having looked through all our dependencies, using that as you suggested, sounds like a good compromise to me.

        Show
        Dennis Lundberg added a comment - Niall, There is a "skip" parameter in maven-remote-resources-plugin that I think can be used to disable the plugin for components that have needs that cannot be accommodated by the plugin. Having looked through all our dependencies, using that as you suggested, sounds like a good compromise to me.
        Hide
        Sebb added a comment -

        This product includes/uses software(s) developed by 'The Apache Software Foundation' (http://jakarta.apache.org)

        should be

        This product includes software developed at
        The Apache Software Foundation (http://www.apache.org)

        [The ASF is not jakarta.a.o]

        Also, there is no point including notices for individual ASF projects - they are all included in the above notice.

        Show
        Sebb added a comment - This product includes/uses software(s) developed by 'The Apache Software Foundation' ( http://jakarta.apache.org ) should be This product includes software developed at The Apache Software Foundation ( http://www.apache.org ) [The ASF is not jakarta.a.o] Also, there is no point including notices for individual ASF projects - they are all included in the above notice.
        Hide
        Dennis Lundberg added a comment -

        Sorry I was looking at "jakarta" in another place in the file.

        The content of the generated NOTICE file is completely dependent on the metadata that is in the repository. This is a double-edged sword. Sometimes it's good and sometimes it's bad. I'll try to explain what happens:

        Validator depends on commons-logging-1.0.4 which has this in its pom:

        <organization>
        <name>The Apache Software Foundation</name>
        <url>http://jakarta.apache.org</url>
        </organization>

        while validator itself inherits this from commons-parent, which inherits from the apache-parent:

        <organization>
        <name>The Apache Software Foundation</name>
        <url>http://www.apache.org/</url>
        </organization>

        The current implementation of the plugin sees these two as different organizations, because it compares both the organization's name and the url. Looking at it from the point of this discussion that seems like a bad thing to do, so I'll open an issue for the plugin to see what others think.

        Show
        Dennis Lundberg added a comment - Sorry I was looking at "jakarta" in another place in the file. The content of the generated NOTICE file is completely dependent on the metadata that is in the repository. This is a double-edged sword. Sometimes it's good and sometimes it's bad. I'll try to explain what happens: Validator depends on commons-logging-1.0.4 which has this in its pom: <organization> <name>The Apache Software Foundation</name> <url> http://jakarta.apache.org </url> </organization> while validator itself inherits this from commons-parent, which inherits from the apache-parent: <organization> <name>The Apache Software Foundation</name> <url> http://www.apache.org/ </url> </organization> The current implementation of the plugin sees these two as different organizations, because it compares both the organization's name and the url. Looking at it from the point of this discussion that seems like a bad thing to do, so I'll open an issue for the plugin to see what others think.
        Hide
        Dennis Lundberg added a comment -
        Show
        Dennis Lundberg added a comment - Created issue is at: http://jira.codehaus.org/browse/MRRESOURCES-30
        Hide
        Niall Pemberton added a comment -

        Dennis,

        Didn't know about the "skip parameter" - wouldn't that mean a component would need to add "rc" and "release" profiles to their poms with config for the plugin in each?

        I had a look at remote resources plugin and found out it does actually ignore a remote resource if a resource of the same name is already specified in the project. Unfortunately its generating a "Notice" file rather than a "Notice.txt" file. I tried renaming the Validator one to "Notice" (and specifying that in the pom) and it picked up the Validator one rather than generating.

        These Notice and License files are being generated by the plugin from velocity templates in the apache-jar-resource-bundle-1.3.jar. If those templates had been named Notice.txt.vm and License.txt.vm (rather than Notice.vm and License.vm) then the plugin would generate Notice.txt and License.txt files (I think) and then any component could either use the generated one or override by adding Notice/License files. That seems a much easier/straight forward way to me.

        So I guess possible solutions (to duplicate entries and people overriding where required) are:

        1) rename the velocity templates in apache-jar-resource-bundle-1.3.jar
        2) create another jar resource bundle containing alternative named license/notice velocity templates
        3) Rename all Notice.txt/License.txt files in commons to Notice/License
        4) Remove the remote resources plugin

        I would quite like to release version 6 of commons-parent so that we have a version with the latest commons-skin (and therefore m2 sites generate with the ApacheCon logo) and to avoid the profile issues I had. How about we opt initially for option 4) and remove the remote-resources-plugin and release commons-parent-6. If/When we have agreement on another solution using that plugin we can put it back in when its ready.

        Show
        Niall Pemberton added a comment - Dennis, Didn't know about the "skip parameter" - wouldn't that mean a component would need to add "rc" and "release" profiles to their poms with config for the plugin in each? I had a look at remote resources plugin and found out it does actually ignore a remote resource if a resource of the same name is already specified in the project. Unfortunately its generating a "Notice" file rather than a "Notice.txt" file. I tried renaming the Validator one to "Notice" (and specifying that in the pom) and it picked up the Validator one rather than generating. These Notice and License files are being generated by the plugin from velocity templates in the apache-jar-resource-bundle-1.3.jar. If those templates had been named Notice.txt.vm and License.txt.vm (rather than Notice.vm and License.vm) then the plugin would generate Notice.txt and License.txt files (I think) and then any component could either use the generated one or override by adding Notice/License files. That seems a much easier/straight forward way to me. So I guess possible solutions (to duplicate entries and people overriding where required) are: 1) rename the velocity templates in apache-jar-resource-bundle-1.3.jar 2) create another jar resource bundle containing alternative named license/notice velocity templates 3) Rename all Notice.txt/License.txt files in commons to Notice/License 4) Remove the remote resources plugin I would quite like to release version 6 of commons-parent so that we have a version with the latest commons-skin (and therefore m2 sites generate with the ApacheCon logo) and to avoid the profile issues I had. How about we opt initially for option 4) and remove the remote-resources-plugin and release commons-parent-6. If/When we have agreement on another solution using that plugin we can put it back in when its ready.
        Hide
        Dennis Lundberg added a comment -

        I've been reading through the links that have been provided regarding license and notice files. Great pointers by the way. I was hoping to find some guidance regarding the naming of these files.

        http://apache.org/dev/apply-license.html#license-file-name

        "Can the LICENSE and NOTICE files be called LICENSE.txt and NOTICE.txt?
        This is permitted. However the preference is that the files be called LICENSE and NOTICE."

        I doubt that the apache-jar-resource-bundle will be altered to go against the recommendation, so will probably not happen. In light of that I'd say that option 3 is the way to go. It is a one-time-change for each component.

        It is nice to not have to configure too much. Just drop in a replacement NOTICE file (or LICENSE file) and it will be used. We must make sure to document it in our build/release docs.

        Show
        Dennis Lundberg added a comment - I've been reading through the links that have been provided regarding license and notice files. Great pointers by the way. I was hoping to find some guidance regarding the naming of these files. http://apache.org/dev/apply-license.html#license-file-name "Can the LICENSE and NOTICE files be called LICENSE.txt and NOTICE.txt? This is permitted. However the preference is that the files be called LICENSE and NOTICE." I doubt that the apache-jar-resource-bundle will be altered to go against the recommendation, so will probably not happen. In light of that I'd say that option 3 is the way to go. It is a one-time-change for each component. It is nice to not have to configure too much. Just drop in a replacement NOTICE file (or LICENSE file) and it will be used. We must make sure to document it in our build/release docs.
        Hide
        Niall Pemberton added a comment -

        Its a bit more work than just renaming the files as we have a mixture and ant, m1 and m2 builds that would also need fixing as a result of the renaming.

        Show
        Niall Pemberton added a comment - Its a bit more work than just renaming the files as we have a mixture and ant, m1 and m2 builds that would also need fixing as a result of the renaming.
        Hide
        Sebb added a comment -

        I think the NOTICE and LICENSE files should be present in SVN at the top-level of each tree.

        This is already the case for the commons projects I looked at.

        I've asked for confirmation of this on the legal discuss list.

        I also think that the N & L files really ought to be manually created to ensure that the requirements are met.

        They don't change very frequently, so regeneration is not an issue.
        And if it is confirmed that the N & L files need to be in SVN, then generating them automatically is pointless.

        Show
        Sebb added a comment - I think the NOTICE and LICENSE files should be present in SVN at the top-level of each tree. This is already the case for the commons projects I looked at. I've asked for confirmation of this on the legal discuss list. I also think that the N & L files really ought to be manually created to ensure that the requirements are met. They don't change very frequently, so regeneration is not an issue. And if it is confirmed that the N & L files need to be in SVN, then generating them automatically is pointless.
        Hide
        Niall Pemberton added a comment -

        Since the majority seem to prefer keeping Notice and License files in svn and the fact that currently the remote resources plugin causes duplicate versions of these files in the jar I'm going to remove the remote resources plugin from the parent pom for now.

        I suggest that if theres a desire by Dennis or others to re-introduce it then the majority need to be convinced first.

        Niall

        P.S. My PoV - I'm fairly neutral either way but I want the current duplication issue resolved and it seems like a whole bunch of work to get to where we already are

        Show
        Niall Pemberton added a comment - Since the majority seem to prefer keeping Notice and License files in svn and the fact that currently the remote resources plugin causes duplicate versions of these files in the jar I'm going to remove the remote resources plugin from the parent pom for now. I suggest that if theres a desire by Dennis or others to re-introduce it then the majority need to be convinced first. Niall P.S. My PoV - I'm fairly neutral either way but I want the current duplication issue resolved and it seems like a whole bunch of work to get to where we already are
        Niall Pemberton committed 608470 (1 file)
        Reviews: none

        COMMONSSITE-21 - remove remote resources plugin

        Hide
        Rahul Akolkar added a comment -

        See also discussion on r608470 on dev list. One pointer below, but should be searchable on any archive using the rev #:

        http://www.nabble.com/svn-commit%3A-r608470----commons-proper-commons-parent-trunk-pom.xml-to14596584.html

        Show
        Rahul Akolkar added a comment - See also discussion on r608470 on dev list. One pointer below, but should be searchable on any archive using the rev #: http://www.nabble.com/svn-commit%3A-r608470----commons-proper-commons-parent-trunk-pom.xml-to14596584.html
        Hide
        Dennis Lundberg added a comment -

        Sebb,

        Did you get any response from the legal discuss list whether we are required to have LICENSE and NOTICE files present in SVN?

        Show
        Dennis Lundberg added a comment - Sebb, Did you get any response from the legal discuss list whether we are required to have LICENSE and NOTICE files present in SVN?
        Hide
        Niall Pemberton added a comment -

        The thread on legal-discuss is here: http://tinyurl.com/22tahy but most of it went off-topic. The two points that seemed relevant to me was 1) there is no such policy for svn, only releases and 2) since any part of the svn tree can be checked out it makes no sense.

        IMO until there is official policy requiring it then there is no legal issue here.

        Show
        Niall Pemberton added a comment - The thread on legal-discuss is here: http://tinyurl.com/22tahy but most of it went off-topic. The two points that seemed relevant to me was 1) there is no such policy for svn, only releases and 2) since any part of the svn tree can be checked out it makes no sense. IMO until there is official policy requiring it then there is no legal issue here.
        Hide
        Simon Kitching added a comment -

        Yes, it does seem that there is no official policy on license/notice files in svn.

        However I'm not convinced by the argument that since any dir can be checked out, there is no logical place to put license/notice files in svn. The svn contents are very clearly structured into modules which are "compilable units". Yes, theoretically someone could check out some leaf directory then claim that the files "have no license information available", but that's a pretty thin excuse. Anyone capable of reusing java code would know where the module ("compilable unit") starts. That is the sensible point at which to check things out - and the license/notice info should be available at that level.

        It's just like putting copyright info inside the cover of a book. It doesn't need to be on every page, because the book is the minimum distributable unit. But in the case of a book series, it is not adequate to put the information in just one of the books in the set. Ok, the analogy isn't perfect because books do not have a "raw" and "compiled" form both available to users. But making source code available on the web via svn is publishing, and so notice/copyright should be available there too, without requiring someone to look into a pom and determine where a maven plugin will pull the information from during a build.

        Show
        Simon Kitching added a comment - Yes, it does seem that there is no official policy on license/notice files in svn. However I'm not convinced by the argument that since any dir can be checked out, there is no logical place to put license/notice files in svn. The svn contents are very clearly structured into modules which are "compilable units". Yes, theoretically someone could check out some leaf directory then claim that the files "have no license information available", but that's a pretty thin excuse. Anyone capable of reusing java code would know where the module ("compilable unit") starts. That is the sensible point at which to check things out - and the license/notice info should be available at that level. It's just like putting copyright info inside the cover of a book. It doesn't need to be on every page, because the book is the minimum distributable unit. But in the case of a book series, it is not adequate to put the information in just one of the books in the set. Ok, the analogy isn't perfect because books do not have a "raw" and "compiled" form both available to users. But making source code available on the web via svn is publishing, and so notice/copyright should be available there too, without requiring someone to look into a pom and determine where a maven plugin will pull the information from during a build.
        Hide
        Niall Pemberton added a comment -

        Well I don't agree, using your analogy svn is the authors draft manuscript and the book=release artifacts. But this is all moot since without a policy IMO objecting to using remote-resource-plugin on legal grounds is not valid. If you're right though then you should take this to members@ and convince them to make it policy.

        Show
        Niall Pemberton added a comment - Well I don't agree, using your analogy svn is the authors draft manuscript and the book=release artifacts. But this is all moot since without a policy IMO objecting to using remote-resource-plugin on legal grounds is not valid. If you're right though then you should take this to members@ and convince them to make it policy.
        Hide
        Dennis Lundberg added a comment - - edited

        Trying to summarize what has been established so far, regarding LICENSE and NOTICE files:

        1. There is no policy regarding whether or these files need to be in svn. Until such a policy exists it is up to the PMC to decide if we want them in svn or not.
        2. Is the PMC decides that these files should be in svn, there is no policy regarding where they should be stored. It is up to the PMC to decide.
        3. There is policy (preference[1]) that the files should be called LICENSE and NOTICE, without the .txt suffix
        4. We agree to disagree on the usefulness of the maven-remote-resources-plugin. Therefor it is up to the component developers to decide if they want to use it or not. If it is used, the PMC can vote -1 on a release if the generated NOTICE file is not deemed good enough.

        And a couple of my own thoughts, not sure if there is consensus about them:

        A. There is a policy [2] that all source files should contain the ASF header, which includes a link to the license. Therefor I do not see the need for a LICENSE file in svn as long as a copy of the ASF license is included in all distributions.

        B. Because the current naming of these files in Commons differ from the ones generated by maven-remote-resources-plugin, there is a possibility of duplicate files being inserted into the distributions. If a component wants to use maven-remote-resources-plugin and want to have these files present in svn, the component should rename the files to LICENSE and NOTICE respectively to avoid this.

        [1] http://apache.org/dev/apply-license.html#license-file-name
        [2] http://apache.org/legal/src-headers.html#headers

        Show
        Dennis Lundberg added a comment - - edited Trying to summarize what has been established so far, regarding LICENSE and NOTICE files: 1. There is no policy regarding whether or these files need to be in svn. Until such a policy exists it is up to the PMC to decide if we want them in svn or not. 2. Is the PMC decides that these files should be in svn, there is no policy regarding where they should be stored. It is up to the PMC to decide. 3. There is policy (preference [1] ) that the files should be called LICENSE and NOTICE, without the .txt suffix 4. We agree to disagree on the usefulness of the maven-remote-resources-plugin. Therefor it is up to the component developers to decide if they want to use it or not. If it is used, the PMC can vote -1 on a release if the generated NOTICE file is not deemed good enough. And a couple of my own thoughts, not sure if there is consensus about them: A. There is a policy [2] that all source files should contain the ASF header, which includes a link to the license. Therefor I do not see the need for a LICENSE file in svn as long as a copy of the ASF license is included in all distributions. B. Because the current naming of these files in Commons differ from the ones generated by maven-remote-resources-plugin, there is a possibility of duplicate files being inserted into the distributions. If a component wants to use maven-remote-resources-plugin and want to have these files present in svn, the component should rename the files to LICENSE and NOTICE respectively to avoid this. [1] http://apache.org/dev/apply-license.html#license-file-name [2] http://apache.org/legal/src-headers.html#headers
        Hide
        Stephen Colebourne added a comment -

        FWIW, I strongly prefer the .txt suffix, as it is much, much more friendly to Windows. The ASF policy was clearly written by a someone who only uses Unix.

        I also believe that the files should be in svn (at the root of the project/component)

        If the result of this is that maven-remote-resources-plugin is broken, well so be it.

        Show
        Stephen Colebourne added a comment - FWIW, I strongly prefer the .txt suffix, as it is much, much more friendly to Windows. The ASF policy was clearly written by a someone who only uses Unix. I also believe that the files should be in svn (at the root of the project/component) If the result of this is that maven-remote-resources-plugin is broken, well so be it.
        Hide
        Niall Pemberton added a comment -

        @Stephen +1

        @Dennis - various thoughts:

        • Commons is currently consistent and IMO should remain that way - both in how N&L are named and handled
        • Renaming N&L files is alot of make-work (ant and m1 builds would break)
        • an alternative Apache N&L package with .txt extenstions would be preferrable to renaming
        • What about source and binary distros? They should contain the same N&L files - how could the generated ones get into those distros?
        • seems like wasted effort to change something that works and is simple and esp. to something that doesn't currently work well
        Show
        Niall Pemberton added a comment - @Stephen +1 @Dennis - various thoughts: Commons is currently consistent and IMO should remain that way - both in how N&L are named and handled Renaming N&L files is alot of make-work (ant and m1 builds would break) an alternative Apache N&L package with .txt extenstions would be preferrable to renaming What about source and binary distros? They should contain the same N&L files - how could the generated ones get into those distros? seems like wasted effort to change something that works and is simple and esp. to something that doesn't currently work well
        Hide
        Dennis Lundberg added a comment -

        OK, we are not getting anywhere right now. I'm trying to find out what we agree upon here. Like I already stated with 4 we disagree on the usefulness of the maven-remote-resources-plugin. My idea was to write down somewhere (commons site) what we agree on, so that we won't have to go through this discussion again.

        So, can we please, please try to find the points where we have consensus. Points 1-4 above was what I thought we had reached consensus about. Just ignore the rest.

        Show
        Dennis Lundberg added a comment - OK, we are not getting anywhere right now. I'm trying to find out what we agree upon here. Like I already stated with 4 we disagree on the usefulness of the maven-remote-resources-plugin. My idea was to write down somewhere (commons site) what we agree on, so that we won't have to go through this discussion again. So, can we please, please try to find the points where we have consensus. Points 1-4 above was what I thought we had reached consensus about. Just ignore the rest.
        Hide
        Jochen Wiedmann added a comment -

        I'd like to note that I do support Dennis' opinion that the maven-remote-resources-plugin should be used, for the following reasons:

        • Commons has a very special policy. In fact, I have been release manager for a multitude
          of ASF projects and subprojects and never found the requirements so heavy. It takes a
          real lot of time and work to accomplish this. A good example is the policy to add LICENSE
          and NOTICE files to the javadoc jar files, which I personally find ridiculous, because (thanks
          god) the javadoc files do not even carry a copyright notice. The more we delegate to
          plugins, the better, it only helps.
        • Most of the open points could very well be fixed by extending or modifying the
          maven-remote-resources-plugin. A good example are the file names. If they are fixed now,
          why should they not be customizable? If the developers do not want to adhere to
          our requirements, then we may very well subclass the plugin.
        • Niall has pointed out, that a consistent behaviour is desirable. That is certainly the
          case. But even a consistent behaviour will require changes at some point. And at that
          point we can also only win by delegating work to a pugin.
        Show
        Jochen Wiedmann added a comment - I'd like to note that I do support Dennis' opinion that the maven-remote-resources-plugin should be used, for the following reasons: Commons has a very special policy. In fact, I have been release manager for a multitude of ASF projects and subprojects and never found the requirements so heavy. It takes a real lot of time and work to accomplish this. A good example is the policy to add LICENSE and NOTICE files to the javadoc jar files, which I personally find ridiculous, because (thanks god) the javadoc files do not even carry a copyright notice. The more we delegate to plugins, the better, it only helps. Most of the open points could very well be fixed by extending or modifying the maven-remote-resources-plugin. A good example are the file names. If they are fixed now, why should they not be customizable? If the developers do not want to adhere to our requirements, then we may very well subclass the plugin. Niall has pointed out, that a consistent behaviour is desirable. That is certainly the case. But even a consistent behaviour will require changes at some point. And at that point we can also only win by delegating work to a pugin.
        Hide
        Niall Pemberton added a comment -

        I posted this issue ticket (probably in the wrong place) :
        http://jira.codehaus.org/browse/MRRESOURCES-31

        Show
        Niall Pemberton added a comment - I posted this issue ticket (probably in the wrong place) : http://jira.codehaus.org/browse/MRRESOURCES-31
        Hide
        Niall Pemberton added a comment -

        @Dennis - sorry for annoying you Dennis - I heres my opnions on your four points:

        • 1) I agree
        • 2) I agree
        • 3) I agree, but it also says ".txt" extensions are acceptable and the best solution IMO for Commons is an alternative bundle as I propose in MRRESOURCES-31[1]
        • 4) I disagree because I don't think any components should be using it until the issues with generation and what to do about the source/binary distros are resolved.

        So is there a way to include the generated NOTICE / LICENSE files in the source and binary distros?

        [1] http://jira.codehaus.org/browse/MRRESOURCES-31

        Show
        Niall Pemberton added a comment - @Dennis - sorry for annoying you Dennis - I heres my opnions on your four points: 1) I agree 2) I agree 3) I agree, but it also says ".txt" extensions are acceptable and the best solution IMO for Commons is an alternative bundle as I propose in MRRESOURCES-31 [1] 4) I disagree because I don't think any components should be using it until the issues with generation and what to do about the source/binary distros are resolved. So is there a way to include the generated NOTICE / LICENSE files in the source and binary distros? [1] http://jira.codehaus.org/browse/MRRESOURCES-31
        Hide
        Dennis Lundberg added a comment -

        Regarding 4. It would be easy enough to add the generated NOTICE and LICENSE file into the assemblies. Just add this snippet to each assembly descriptor:

            <fileSet>
              <directory>target/maven-shared-archive-resources/META-INF</directory>
              <outputDirectory></outputDirectory>
              <includes>
                <include>LICENSE</include>
                <include>NOTICE</include>
              </includes>
            </fileSet>
        
        Show
        Dennis Lundberg added a comment - Regarding 4. It would be easy enough to add the generated NOTICE and LICENSE file into the assemblies. Just add this snippet to each assembly descriptor: <fileSet> <directory> target/maven-shared-archive-resources/META-INF </directory> <outputDirectory> </outputDirectory> <includes> <include> LICENSE </include> <include> NOTICE </include> </includes> </fileSet>
        Hide
        Mark Struberg added a comment -

        foooolks, can we cleanup all such issues?

        Show
        Mark Struberg added a comment - foooolks, can we cleanup all such issues?
        Hide
        Phil Steitz added a comment -

        Too bad we have lost the svn commit tracking in JIRA; otherwise we could confirm this has been "fixed." Unless anyone pipes up to disagree, I would say close this.

        Show
        Phil Steitz added a comment - Too bad we have lost the svn commit tracking in JIRA; otherwise we could confirm this has been "fixed." Unless anyone pipes up to disagree, I would say close this.
        Hide
        Ate Douma added a comment -

        Seems nobody piped up to disagree. Time to close?

        Show
        Ate Douma added a comment - Seems nobody piped up to disagree. Time to close?
        Hide
        Niall Pemberton added a comment -

        closing

        Show
        Niall Pemberton added a comment - closing
        Niall Pemberton made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Niall Pemberton
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development