Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6, 4.0-ALPHA
    • Component/s: general/build
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Currently once the pom.xml's are in place, its hard to get them out. Having them can be a little trappy when you're trying to debug the bug. We should facilitate their removal during clean.

      1. LUCENE-3944.patch
        11 kB
        Steve Rowe
      2. LUCENE-3944.patch
        10 kB
        Steve Rowe
      3. LUCENE-3944.patch
        10 kB
        Steve Rowe

        Activity

        Hide
        Robert Muir added a comment -

        This is currently pretty crazy, we need to fix this to have a reproducible build.

        the problem is 'prepare-release' can pass or fail, depending upon the presence of svn-ignore'd pom.xmls scattered about the source tree.
        on top of this, some modules (at least solr/contrib/langid) rely upon the fact you have previous populated svn-ignore'd pom.xml's elsewhere in solr first... but the fact that these are in the source tree svn:ignore'd and not removed on clean basically makes it nearly impossible to debug.

        Show
        Robert Muir added a comment - This is currently pretty crazy, we need to fix this to have a reproducible build. the problem is 'prepare-release' can pass or fail, depending upon the presence of svn-ignore'd pom.xmls scattered about the source tree. on top of this, some modules (at least solr/contrib/langid) rely upon the fact you have previous populated svn-ignore'd pom.xml's elsewhere in solr first... but the fact that these are in the source tree svn:ignore'd and not removed on clean basically makes it nearly impossible to debug.
        Hide
        Robert Muir added a comment -

        steps to reproduce current craziness:

        cd solr/
        ant generate-maven-artifacts
        cd ..
        ant clean clean-jars
        cd solr/contrib/langid
        ant dist-maven <-- works
        now go back to the root
        find . -name pom.xml | xargs rm -f
        ant clean clean-jars
        ant cd solr/contrib/langid
        ant dist-maven <-- fails

        Show
        Robert Muir added a comment - steps to reproduce current craziness: cd solr/ ant generate-maven-artifacts cd .. ant clean clean-jars cd solr/contrib/langid ant dist-maven <-- works now go back to the root find . -name pom.xml | xargs rm -f ant clean clean-jars ant cd solr/contrib/langid ant dist-maven <-- fails
        Hide
        Robert Muir added a comment -

        I don't really think ant clean should be nuking things from the source tree,
        instead, I think just like tests (SOLR-3268), 'dist-maven' should not modify the
        source tree here but only build and dist with this build output.

        Show
        Robert Muir added a comment - I don't really think ant clean should be nuking things from the source tree, instead, I think just like tests ( SOLR-3268 ), 'dist-maven' should not modify the source tree here but only build and dist with this build output.
        Hide
        Steve Rowe added a comment -

        Patch implementing a new target filter-pom-templates, which places the POMs under lucene/build/poms/. dist-maven pulls POMs from this location instead of from each module's source directory. generate-maven-artifacts under lucene/ seems to work; I haven't tested much else.

        There is an Ant problem, though: I have attempted to override dist-maven in solr/common-build.xml, but it's being ignored in Solr modules, which are instead using the dist-maven definition in lucene/common-build.xml. My reading of the Ant docs makes me think this may be the intended behavior: only the running build file's target definitions override those that are imported.

        Show
        Steve Rowe added a comment - Patch implementing a new target filter-pom-templates , which places the POMs under lucene/build/poms/ . dist-maven pulls POMs from this location instead of from each module's source directory. generate-maven-artifacts under lucene/ seems to work; I haven't tested much else. There is an Ant problem, though: I have attempted to override dist-maven in solr/common-build.xml , but it's being ignored in Solr modules, which are instead using the dist-maven definition in lucene/common-build.xml . My reading of the Ant docs makes me think this may be the intended behavior: only the running build file's target definitions override those that are imported.
        Hide
        Steve Rowe added a comment -

        New trunk patch.

        Robert pointed out on #lucene IRC that the Ant problem (Solr contrib ignoring an overridden dist-maven target definition in solr/common-build.xml) was as a result of the dist-maven specialization in solr/contrib/langid, which depended on the lucene definition instead of the overriding solr definition. This is fixed in this version of the patch.

        Also fixed in this version of the patch: a typo in the Solr definition of dist-maven that prevented it from finding the appropriate POM.

        ant generate-maven-artifacts now works (succeeds anyway and puts the right stuff in the right place) under solr/, lucene/, and modules/.

        I plan on doing more testing before committing, and then backporting to branch_3x.

        Show
        Steve Rowe added a comment - New trunk patch. Robert pointed out on #lucene IRC that the Ant problem (Solr contrib ignoring an overridden dist-maven target definition in solr/common-build.xml ) was as a result of the dist-maven specialization in solr/contrib/langid , which depended on the lucene definition instead of the overriding solr definition. This is fixed in this version of the patch. Also fixed in this version of the patch: a typo in the Solr definition of dist-maven that prevented it from finding the appropriate POM. ant generate-maven-artifacts now works (succeeds anyway and puts the right stuff in the right place) under solr/ , lucene/ , and modules/ . I plan on doing more testing before committing, and then backporting to branch_3x.
        Hide
        Robert Muir added a comment -

        Patch works for me. A nice improvement in the future would be if solr's common-build
        defined filtered.pom.templates.dir to be under its own solr/build (rather than lucene's),
        but that doesnt need to be fixed here... this is really nice since it doesnt do
        this stuff under the source tree. Thanks for working on this!

        Show
        Robert Muir added a comment - Patch works for me. A nice improvement in the future would be if solr's common-build defined filtered.pom.templates.dir to be under its own solr/build (rather than lucene's), but that doesnt need to be fixed here... this is really nice since it doesnt do this stuff under the source tree. Thanks for working on this!
        Hide
        Steve Rowe added a comment -

        A nice improvement in the future would be if solr's common-build defined filtered.pom.templates.dir to be under its own solr/build (rather than lucene's)

        Where would modules/ POMs go - each under its own build directory?

        If we ever try to do this, I think Maven Ant Tasks' deploy target needs to be able to access the parent and grandparent POMs, which (I think) means either putting them into the user's local maven repository, or putting them at the relative location given in the parent POM section of each POM. Maybe in order to satisfy this the grandparent POM could be put in both places (or all places if modules/ need their own)?

        Show
        Steve Rowe added a comment - A nice improvement in the future would be if solr's common-build defined filtered.pom.templates.dir to be under its own solr/build (rather than lucene's) Where would modules/ POMs go - each under its own build directory? If we ever try to do this, I think Maven Ant Tasks' deploy target needs to be able to access the parent and grandparent POMs, which (I think) means either putting them into the user's local maven repository, or putting them at the relative location given in the parent POM section of each POM. Maybe in order to satisfy this the grandparent POM could be put in both places (or all places if modules/ need their own)?
        Hide
        Robert Muir added a comment -

        Currently modules/ is bogus in the lucene build, some stuff goes in modules/xxx/build, some in lucene/build/....

        So its a larger problem, thats why I think its a nicer improvement. As far as duplicating any poms in both locations, i think thats ok.

        When we release, currently lucene and solr are different releases (you run ant prepare-release for both), so they are independent artifacts.

        But yeah, all of this can be a future improvement for a future issue.

        +1 for going with this patch (or some refinement on it based on testing, i only did light testing) so that its not changing the source tree.

        Show
        Robert Muir added a comment - Currently modules/ is bogus in the lucene build, some stuff goes in modules/xxx/build, some in lucene/build/.... So its a larger problem, thats why I think its a nicer improvement. As far as duplicating any poms in both locations, i think thats ok. When we release, currently lucene and solr are different releases (you run ant prepare-release for both), so they are independent artifacts. But yeah, all of this can be a future improvement for a future issue. +1 for going with this patch (or some refinement on it based on testing, i only did light testing) so that its not changing the source tree.
        Hide
        Dawid Weiss added a comment -

        I think Maven Ant Tasks' deploy target needs to be able to access the parent and grandparent POMs, which (I think) means either putting them into the user's local maven repository, or putting them at the relative location given in the parent POM section of each POM.

        I just recently peeked at Apache ANT's source distribution and this seems to be done this way (separate folder structure just for POMs with relative refs).

        Show
        Dawid Weiss added a comment - I think Maven Ant Tasks' deploy target needs to be able to access the parent and grandparent POMs, which (I think) means either putting them into the user's local maven repository, or putting them at the relative location given in the parent POM section of each POM. I just recently peeked at Apache ANT's source distribution and this seems to be done this way (separate folder structure just for POMs with relative refs).
        Hide
        Steve Rowe added a comment -

        I just recently peeked at Apache ANT's source distribution and this seems to be done this way (separate folder structure just for POMs with relative refs).

        On #lucene IRC last night and this morning, Chris Male pointed out Ant's POM setup, and said that he's working on enabling exactly the same setup for Lucene/Solr. I believe the new filter-pom-templates target included in my patches on this issue could be part of such a change.

        Show
        Steve Rowe added a comment - I just recently peeked at Apache ANT's source distribution and this seems to be done this way (separate folder structure just for POMs with relative refs). On #lucene IRC last night and this morning, Chris Male pointed out Ant's POM setup, and said that he's working on enabling exactly the same setup for Lucene/Solr. I believe the new filter-pom-templates target included in my patches on this issue could be part of such a change.
        Hide
        Steve Rowe added a comment -

        Final trunk patch. I ran ant dist-maven on all directories containing a build.xml file. (Some failed, but in all cases that was due to not having a dist-maven target.)

        The changes from the last iteration:

        • includes CHANGES entry
        • removed full Solr duplicate of dist-maven (only dependencies were different from the Lucene version) in favor of new target dist-maven-common, which is depended on by both Lucene's and Solr's dist-maven targets
        • fixed dist-maven in solr/webapp/ to depend on install-maven-tasks
        • fixed top-level generate-maven-artifacts so that it functions properly.

        Committing to trunk shortly, then backporting to branch_3x.

        Show
        Steve Rowe added a comment - Final trunk patch. I ran ant dist-maven on all directories containing a build.xml file. (Some failed, but in all cases that was due to not having a dist-maven target.) The changes from the last iteration: includes CHANGES entry removed full Solr duplicate of dist-maven (only dependencies were different from the Lucene version) in favor of new target dist-maven-common , which is depended on by both Lucene's and Solr's dist-maven targets fixed dist-maven in solr/webapp/ to depend on install-maven-tasks fixed top-level generate-maven-artifacts so that it functions properly. Committing to trunk shortly, then backporting to branch_3x.
        Hide
        Steve Rowe added a comment -

        Committed to trunk and branch_3x.

        Show
        Steve Rowe added a comment - Committed to trunk and branch_3x.

          People

          • Assignee:
            Steve Rowe
            Reporter:
            Chris Male
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development