Cocoon
  1. Cocoon
  2. COCOON-2330

Release separate source and dependencies packages

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.11
    • Fix Version/s: 2.1.12
    • Component/s: - Build System: Ant
    • Labels:
      None

      Description

      As recently reported by David Corssley [1] the 2.1.X release must comply with ASF policies about releases, in particular providing a separate source code package (without any external JAR file inside) and dependencies package (with all external JAR files) for distribution.

      [1] http://markmail.org/message/f6rm2ca35eiraf7i

        Activity

        Hide
        Francesco Chicchiriccò added a comment -
        +1

        I've checked again the legal stuff, built with './build.s dist', checked the resulting zip / tar.gz, expanded in a new directory, tried to build src without deps and finally assembled all together with a successful build.

        Nice job!
        Show
        Francesco Chicchiriccò added a comment - +1 I've checked again the legal stuff, built with './build.s dist', checked the resulting zip / tar.gz, expanded in a new directory, tried to build src without deps and finally assembled all together with a successful build. Nice job!
        Hide
        Cédric Damioli added a comment -
        I just committed the removal of old legal/*.license.txt as well as the new dist-build.xml reflecting the changes

        David, Fransesco, are you ok with that ? (BTW, thank you David for fixing the licence file)

        If so, I think we can close this issue.
        Show
        Cédric Damioli added a comment - I just committed the removal of old legal/*.license.txt as well as the new dist-build.xml reflecting the changes David, Fransesco, are you ok with that ? (BTW, thank you David for fixing the licence file) If so, I think we can close this issue.
        Hide
        Francesco Chicchiriccò added a comment -
        They looks good, +1, nice job!
        The road to 2.1.12 seems to be shortening...
        Show
        Francesco Chicchiriccò added a comment - They looks good, +1, nice job! The road to 2.1.12 seems to be shortening...
        Hide
        Cédric Damioli added a comment -
        I've committed legal/NOTICE.txt and legal/LICENSE.txt containing the merge of licensing information.

        Before removing old-fashioned license files and adapting the build, could you please tell me if you agree with what I've done ?
        Show
        Cédric Damioli added a comment - I've committed legal/NOTICE.txt and legal/LICENSE.txt containing the merge of licensing information. Before removing old-fashioned license files and adapting the build, could you please tell me if you agree with what I've done ?
        Hide
        Francesco Chicchiriccò added a comment -
        Not completely: I still believe we need to separate LICENSEs and NOTICEs.

        I'd change your latest proposal as:

         * use current LICENSE.txt and NOTICE.txt for src package

         * create legal-deps/LICENSE.txt and legal-deps/NOTICE.txt being the merge of all legal/*.license.txt under the following conditions:

          1. do not repeat the same license more than once: if the EPL license is already there for dependency X, just say "dependency Y released under EPL, see above"

          2. split the content of current legal/$1.txt between LICENSE and NOTICE; for example the content of 'jfor-0.7.1.jar.license.txt' will become

        NOTICE.txt
        """
          This product includes software developed by the jfor project.
         Copyright (c) 2002 by the jfor project. All rights reserved.
        """

        LICENSE.txt
        """
        (the rest of jfor-0.7.1.jar.license.txt's content)
        """

        ...and finally remove the whole legal subdir.

        WDYT?
        Show
        Francesco Chicchiriccò added a comment - Not completely: I still believe we need to separate LICENSEs and NOTICEs. I'd change your latest proposal as:  * use current LICENSE.txt and NOTICE.txt for src package  * create legal-deps/LICENSE.txt and legal-deps/NOTICE.txt being the merge of all legal/*.license.txt under the following conditions:   1. do not repeat the same license more than once: if the EPL license is already there for dependency X, just say "dependency Y released under EPL, see above"   2. split the content of current legal/$1.txt between LICENSE and NOTICE; for example the content of 'jfor-0.7.1.jar.license.txt' will become NOTICE.txt """   This product includes software developed by the jfor project.  Copyright (c) 2002 by the jfor project. All rights reserved. """ LICENSE.txt """ (the rest of jfor-0.7.1.jar.license.txt's content) """ ...and finally remove the whole legal subdir. WDYT?
        Hide
        Cédric Damioli added a comment -
        I'm fully ok with what you said, but even after reading http://www.apache.org/dev/licensing-howto.html#mod-notice, I still don't understand what I'm supposed to put in the "rich" NOTICE file. It seems that my understanding of legal english is inadequate ! :)
        I don't know how to understand "Aside from Apache-licensed dependencies which supply NOTICE files of their own, it is uncommon for a dependency to require additions to NOTICE.". Does it mean that for Apache-licensed dependencies, NOTICE files has to be merged ?
        In your ManifoldCF example, the "rich" NOTICE file is the same that the "light" one, and Apache-licensed dependencies are not listed in that particular file, but in the LICENSE file.

        What I propose concretely to do:
         - leave NOTICE.txt as is and include in both "src" and "deps" packages : it will be at the root of both packages
         - leave LICENSE.txt as is in the "src" package
         - create a legal/LICENSE.txt being the merge of all legal/*.license.txt based on the model of ManifoldCF. At build time, this file will be put at the root of the the "deps" package.
         - remove all legal/*.license.txt

        do you agree ?
        Show
        Cédric Damioli added a comment - I'm fully ok with what you said, but even after reading http://www.apache.org/dev/licensing-howto.html#mod-notice, I still don't understand what I'm supposed to put in the "rich" NOTICE file. It seems that my understanding of legal english is inadequate ! :) I don't know how to understand "Aside from Apache-licensed dependencies which supply NOTICE files of their own, it is uncommon for a dependency to require additions to NOTICE.". Does it mean that for Apache-licensed dependencies, NOTICE files has to be merged ? In your ManifoldCF example, the "rich" NOTICE file is the same that the "light" one, and Apache-licensed dependencies are not listed in that particular file, but in the LICENSE file. What I propose concretely to do:  - leave NOTICE.txt as is and include in both "src" and "deps" packages : it will be at the root of both packages  - leave LICENSE.txt as is in the "src" package  - create a legal/LICENSE.txt being the merge of all legal/*.license.txt based on the model of ManifoldCF. At build time, this file will be put at the root of the the "deps" package.  - remove all legal/*.license.txt do you agree ?
        Hide
        Francesco Chicchiriccò added a comment -
        As stated in [1], "LICENSE and NOTICE belong at the top level of the source tree. They may be named LICENSE.txt and NOTICE.txt, but the bare names are preferred."

        Moreover, in [2] "The NOTICE file is described in section 4.4 of the Apache License version 2.0. It presence is not mandated by the license itself, but by ASF policy."

        Basically, this means that LICENSE and NOTICE *must* be delivered at top level of every package release. So, both -src and -deps in our case.

        The legal/* stuff is something that I guess was established before this ruling was stated clearly in its definitive form.


        If you take a look at ManifoldCF release files [3] and [4] you can see how they solved the same issue: the -src package comes with "simple" LICENSE and NOTICE at root level + lib-license/ subdir containing "rich" LICENSE and NOTICE while the -lib (i.e. the -deps) packages comes with "rich" LICENSE and NOTICE at root level.

        This means of course that
         1. when extracting the -src package, LICENSE and NOTICE are "simple"
         2. when extracting the -deps package alone, LICENSE and NOTICE are "rich"
         3. when extracting the -deps package over the -src package, LICENSE and NOTICE are "rich", as it should be since the "rich" version fully includes the "simple" version

        [1] http://www.apache.org/dev/licensing-howto.html#source-tree-location
        [2] http://www.apache.org/dev/licensing-howto.html#overview-of-files
        [3] http://www.apache.org/dyn/closer.cgi/manifoldcf/apache-manifoldcf-1.1-src.zip
        [4] http://www.apache.org/dyn/closer.cgi/manifoldcf/apache-manifoldcf-1.1-lib.zip
        Show
        Francesco Chicchiriccò added a comment - As stated in [1], "LICENSE and NOTICE belong at the top level of the source tree. They may be named LICENSE.txt and NOTICE.txt, but the bare names are preferred." Moreover, in [2] "The NOTICE file is described in section 4.4 of the Apache License version 2.0. It presence is not mandated by the license itself, but by ASF policy." Basically, this means that LICENSE and NOTICE *must* be delivered at top level of every package release. So, both -src and -deps in our case. The legal/* stuff is something that I guess was established before this ruling was stated clearly in its definitive form. If you take a look at ManifoldCF release files [3] and [4] you can see how they solved the same issue: the -src package comes with "simple" LICENSE and NOTICE at root level + lib-license/ subdir containing "rich" LICENSE and NOTICE while the -lib (i.e. the -deps) packages comes with "rich" LICENSE and NOTICE at root level. This means of course that  1. when extracting the -src package, LICENSE and NOTICE are "simple"  2. when extracting the -deps package alone, LICENSE and NOTICE are "rich"  3. when extracting the -deps package over the -src package, LICENSE and NOTICE are "rich", as it should be since the "rich" version fully includes the "simple" version [1] http://www.apache.org/dev/licensing-howto.html#source-tree-location [2] http://www.apache.org/dev/licensing-howto.html#overview-of-files [3] http://www.apache.org/dyn/closer.cgi/manifoldcf/apache-manifoldcf-1.1-src.zip [4] http://www.apache.org/dyn/closer.cgi/manifoldcf/apache-manifoldcf-1.1-lib.zip
        Hide
        Cédric Damioli added a comment -
        What are we supposed to do right now ?

        For the LICENSE file :
        Merge all legal/*.licence.txt in a single LICENSE file in the "deps" package ? The problem is that the LICENSE file in the "deps" package will overwrite the one in the "src" package...
        Leave as is, with our legal/ directory containing all relevant license information ?

        For the NOTICE file :
        By definition, the "src" package only contains ASF-licensed files, doesn't it ?
        For the "deps" package, should we merge the credits in the CREDITS.TXT and the few notices present in the legal/ directory ?
        Show
        Cédric Damioli added a comment - What are we supposed to do right now ? For the LICENSE file : Merge all legal/*.licence.txt in a single LICENSE file in the "deps" package ? The problem is that the LICENSE file in the "deps" package will overwrite the one in the "src" package... Leave as is, with our legal/ directory containing all relevant license information ? For the NOTICE file : By definition, the "src" package only contains ASF-licensed files, doesn't it ? For the "deps" package, should we merge the credits in the CREDITS.TXT and the few notices present in the legal/ directory ?
        Hide
        David Crossley added a comment - - edited
        Cocoon-2.1 has been using a file called CREDITS.txt which also contains the stuff that belongs in NOTICE.txt file. Some of it might be relevant for the "src" package NOTICE file, while the rest belongs in the "deps" package NOTICE file.

        There is a new document which attempts to explain both NOTICE and LICENSE.
        http://www.apache.org/dev/licensing-howto.html
        Show
        David Crossley added a comment - - edited Cocoon-2.1 has been using a file called CREDITS.txt which also contains the stuff that belongs in NOTICE.txt file. Some of it might be relevant for the "src" package NOTICE file, while the rest belongs in the "deps" package NOTICE file. There is a new document which attempts to explain both NOTICE and LICENSE. http://www.apache.org/dev/licensing-howto.html
        Hide
        Cédric Damioli added a comment -
        I've committed a change to build.bat and build.sh, checking for the presence of tools/ant/lib/ant.jar

        The -deps package now includes NOTICE.txt and legal/ subdir

        You may find the related dist zip files at http://people.apache.org/~cdamioli/cocoon-2.1.12-rc2-src.zip and http://people.apache.org/~cdamioli/cocoon-2.1.12-rc2-deps.zip

        Could someone with better English writing skills than me update the README.txt to explain the new behaviour ?
        Show
        Cédric Damioli added a comment - I've committed a change to build.bat and build.sh, checking for the presence of tools/ant/lib/ant.jar The -deps package now includes NOTICE.txt and legal/ subdir You may find the related dist zip files at http://people.apache.org/~cdamioli/cocoon-2.1.12-rc2-src.zip and http://people.apache.org/~cdamioli/cocoon-2.1.12-rc2-deps.zip Could someone with better English writing skills than me update the README.txt to explain the new behaviour ?
        Hide
        Cédric Damioli added a comment -
        Ok for the NOTICE file, which in your example is the same as for the src package

        For the LICENSE file, we already have the legal directory, containing all license information for each third party lib. Is it necessary to have a single file with exactly the same information ?
        Show
        Cédric Damioli added a comment - Ok for the NOTICE file, which in your example is the same as for the src package For the LICENSE file, we already have the legal directory, containing all license information for each third party lib. Is it necessary to have a single file with exactly the same information ?
        Hide
        Francesco Chicchiriccò added a comment -
        Cédric, I've been watching several question like yours while Syncope was still in the incubator.

        In our case, LICENSE & NOTICE for the source package are already fine (just the ones currently on the root of source tree).

        For the -deps package, however, we need to provide different LICENSE & NOTICE that will report license and copyright information for each of the JAR files contained.

        FYI, take a look at the discussion [1] about Apache ManifoldCF [2] which releases for download a source package, a binary package (not our case) and a "lib" package whose purpose is the same of the -deps package we are discussing here.

        As an example of the content of our -deps LICENSE and NOTICE files, you can take a look at LICENSE and NOTICE files contained in [3].

        [1] http://old.nabble.com/Binary-dependencies-in-source-releases-(Was%3A--VOTE--Release-ManifoldCF-0.5-incubating,-RC0)-td33579703.html
        [2] http://manifoldcf.apache.org/en_US/download.html
        [3] http://www.apache.org/dyn/closer.cgi/manifoldcf/apache-manifoldcf-1.0.1-lib.tar.gz
        Show
        Francesco Chicchiriccò added a comment - Cédric, I've been watching several question like yours while Syncope was still in the incubator. In our case, LICENSE & NOTICE for the source package are already fine (just the ones currently on the root of source tree). For the -deps package, however, we need to provide different LICENSE & NOTICE that will report license and copyright information for each of the JAR files contained. FYI, take a look at the discussion [1] about Apache ManifoldCF [2] which releases for download a source package, a binary package (not our case) and a "lib" package whose purpose is the same of the -deps package we are discussing here. As an example of the content of our -deps LICENSE and NOTICE files, you can take a look at LICENSE and NOTICE files contained in [3]. [1] http://old.nabble.com/Binary-dependencies-in-source-releases-(Was%3A--VOTE--Release-ManifoldCF-0.5-incubating,-RC0)-td33579703.html [2] http://manifoldcf.apache.org/en_US/download.html [3] http://www.apache.org/dyn/closer.cgi/manifoldcf/apache-manifoldcf-1.0.1-lib.tar.gz
        Hide
        Cédric Damioli added a comment -
        Is there really a need for a LICENSE file in the -deps package, which is provided only for convenience ?
        All license files related to the included jars are already provided in the legal directory.

        Same for NOTICE file. What should we say in that file in the -deps package ?
        Show
        Cédric Damioli added a comment - Is there really a need for a LICENSE file in the -deps package, which is provided only for convenience ? All license files related to the included jars are already provided in the legal directory. Same for NOTICE file. What should we say in that file in the -deps package ?
        Hide
        Francesco Chicchiriccò added a comment -
        As reported by David [1] there is of course the need of a LICENSE and NOTICE file in both packages.

        [1] http://markmail.org/message/ijmy6nzd2k4yzghn
        Show
        Francesco Chicchiriccò added a comment - As reported by David [1] there is of course the need of a LICENSE and NOTICE file in both packages. [1] http://markmail.org/message/ijmy6nzd2k4yzghn
        Hide
        Francesco Chicchiriccò added a comment -
        Just checked out from SVN and issued './build.sh dist'. This ends with the dist directory populated with:

        * cocoon-2.1.12-dev-deps.tar.gz
        * cocoon-2.1.12-dev-deps.zip
        * cocoon-2.1.12-dev-src.tar.gz
        * cocoon-2.1.12-dev-src.zip

        That's fine.

        About -src:
         * I agree with David, the legal/ subdir should move to -deps
         * besides this legal subdir, I think the rest of the package is just fine (even though the dir name seems misleading, even the tools/bin directory only contains text scripts)
         * ./build.sh gives
        Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tools/ant/launch/Launcher
        Caused by: java.lang.ClassNotFoundException: org.apache.tools.ant.launch.Launcher
        at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
        Could not find the main class: org.apache.tools.ant.launch.Launcher. Program will exit.

        My opinion is then that if we move legal/ from -src to -deps and fix the build.sh / build.bat so that they run only with all -deps at their own place, we can close this issue.
        Show
        Francesco Chicchiriccò added a comment - Just checked out from SVN and issued './build.sh dist'. This ends with the dist directory populated with: * cocoon-2.1.12-dev-deps.tar.gz * cocoon-2.1.12-dev-deps.zip * cocoon-2.1.12-dev-src.tar.gz * cocoon-2.1.12-dev-src.zip That's fine. About -src:  * I agree with David, the legal/ subdir should move to -deps  * besides this legal subdir, I think the rest of the package is just fine (even though the dir name seems misleading, even the tools/bin directory only contains text scripts)  * ./build.sh gives Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tools/ant/launch/Launcher Caused by: java.lang.ClassNotFoundException: org.apache.tools.ant.launch.Launcher at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Could not find the main class: org.apache.tools.ant.launch.Launcher. Program will exit. My opinion is then that if we move legal/ from -src to -deps and fix the build.sh / build.bat so that they run only with all -deps at their own place, we can close this issue.
        Hide
        Cédric Damioli added a comment -
        ok for the legal directory
        the 'ls legal | grep -v jar' returns 4 lines :

         - the midi stuff seems ok as they refer to actual Cocoon sources based on XMidi material
         - the htmlarea component is used by the forms block and its resources have been merged into Cocoon sources. I don't know what to do with this one
         - spark correspond to lib/optional/spark-0.2.jar

        I also wonder about tools/bin, which is basically the bin directory of ant. Should we also exclude it from the -src package ?

        And yes you're right, I've added a check for libs inside the build, but if the build itself cannot launch, we have to add a message
        Show
        Cédric Damioli added a comment - ok for the legal directory the 'ls legal | grep -v jar' returns 4 lines :  - the midi stuff seems ok as they refer to actual Cocoon sources based on XMidi material  - the htmlarea component is used by the forms block and its resources have been merged into Cocoon sources. I don't know what to do with this one  - spark correspond to lib/optional/spark-0.2.jar I also wonder about tools/bin, which is basically the bin directory of ant. Should we also exclude it from the -src package ? And yes you're right, I've added a check for libs inside the build, but if the build itself cannot launch, we have to add a message
        Hide
        David Crossley added a comment -
        Probably need to include the "legal" directory into the "-deps" package, as those are the licenses that accompany the jar files. Doing 'ls legal | grep -v \.jar' shows some others that might need investigation.

        I tried building with only the "-src" package. The build failed before getting to any deps verification step or "check-jars" step. Perhaps because missing the Ant jars. The build.sh script might need to check for the libs before starting.

        Show
        David Crossley added a comment - Probably need to include the "legal" directory into the "-deps" package, as those are the licenses that accompany the jar files. Doing 'ls legal | grep -v \.jar' shows some others that might need investigation. I tried building with only the "-src" package. The build failed before getting to any deps verification step or "check-jars" step. Perhaps because missing the Ant jars. The build.sh script might need to check for the libs before starting.
        Hide
        Cédric Damioli added a comment -
        It is of course theoretically possible but is it worth doing it for Cocoon 2.1.12 ?
        Newer branches are maven based so the issue concerns only 2.1.x
        Show
        Cédric Damioli added a comment - It is of course theoretically possible but is it worth doing it for Cocoon 2.1.12 ? Newer branches are maven based so the issue concerns only 2.1.x
        Hide
        Daniele Madama added a comment -
        Cocoon 2.1.x is ant based, is possible to integrate Ivy and let him to download the jars in lib directory (or download normally and then with an ant task move in the right place)?
        Show
        Daniele Madama added a comment - Cocoon 2.1.x is ant based, is possible to integrate Ivy and let him to download the jars in lib directory (or download normally and then with an ant task move in the right place)?
        Hide
        Cédric Damioli added a comment -
        Do you think we should fail if lib directories are not present, or simply display a warning message and let the build fail with a lot of javac errors ?
        Show
        Cédric Damioli added a comment - Do you think we should fail if lib directories are not present, or simply display a warning message and let the build fail with a lot of javac errors ?
        Hide
        Cédric Damioli added a comment -
        The .class files in tools/loader and tools/anttasks are created by the build process so there are no issues with them.

        Does all this stuff only concern JAR files or also other files copied from other projects (eg tools/bin are ant scripts) ?
        Show
        Cédric Damioli added a comment - The .class files in tools/loader and tools/anttasks are created by the build process so there are no issues with them. Does all this stuff only concern JAR files or also other files copied from other projects (eg tools/bin are ant scripts) ?
        Hide
        Francesco Chicchiriccò added a comment -
        JAR files are present under

        lib/core
        lib/endorsed
        lib/optional
        tools/lib
        tools/instrumentation
        tools/jetty/lib

        There are also some .class under

        tools/loader
        tools/anttasks


        I am not familiar at all with Cocoon's ant build, but would it be enough to simply package all the stuff above in a separate ZIP file and make the build fail when any of these is missing from the local source directory? Or is it already like this? (I've noticed the check-jars stuff...)
        Show
        Francesco Chicchiriccò added a comment - JAR files are present under lib/core lib/endorsed lib/optional tools/lib tools/instrumentation tools/jetty/lib There are also some .class under tools/loader tools/anttasks I am not familiar at all with Cocoon's ant build, but would it be enough to simply package all the stuff above in a separate ZIP file and make the build fail when any of these is missing from the local source directory? Or is it already like this? (I've noticed the check-jars stuff...)

          People

          • Assignee:
            Unassigned
            Reporter:
            Francesco Chicchiriccò
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development