Lucene - Core
  1. Lucene - Core
  2. LUCENE-2774

ant generate-maven-artifacts target broken for contrib

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.1, 4.0-ALPHA
    • Fix Version/s: None
    • Component/s: general/build
    • Labels:
      None

      Description

      When executing 'ant generate-maven-artifacts' from a pristine checkout of branch_3x/lucene or trunk/lucene the following error is encountered:

      dist-maven:
           [copy] Copying 1 file to /home/drew/lucene/branch_3x/lucene/build/contrib/analyzers/common
      [artifact:install-provider] Installing provider: org.apache.maven.wagon:wagon-ssh:jar:1.0-beta-2:runtime
      [artifact:pom] An error has occurred while processing the Maven artifact tasks.
      [artifact:pom]  Diagnosis:
      [artifact:pom] 
      [artifact:pom] Unable to initialize POM pom.xml.template: Cannot find parent: org.apache.lucene:lucene-contrib for project: org.apache.lucene:lucene-analyzers:jar:3.1-SNAPSHOT for project org.apache.lucene:lucene-analyzers:jar:3.1-SNAPSHOT
      [artifact:pom] Unable to download the artifact from any repository
      

      The contrib portion of the ant build is executed in a subant task which does not pick up the pom definitions for lucene-parent and lucene-contrib from the main build.xml, so the lucene-parent and lucene-controb poms must be loaded specifically as a part of the contrib build using the artifact:pom task.

      1. LUCENE-2774.patch
        0.6 kB
        Drew Farris

        Activity

        Hide
        Drew Farris added a comment -

        Attached a trivial patch that applies cleanly against both the 3.x branch and trunk.

        Show
        Drew Farris added a comment - Attached a trivial patch that applies cleanly against both the 3.x branch and trunk.
        Hide
        Steve Rowe added a comment -

        Hi Drew,

        I tried running ant generate-maven-artifacts without your patch, and I see the same failure as you do.

        (Actually, when I first tried, with Ant 1.8.1, it wasn't able to find the maven ant tasks jar (the most recent) that I had placed in $ANT_HOME/lib/ - I downgraded to Ant 1.7.0 and everything went fine. Apparently Ant 1.8.X no longer pays attention to jars in $ANT_HOME/lib/?)

        However, after applying your patch, the build still fails on trunk (haven't tried branch_3x yet), though not with the same failure.

        (Ant 1.7.0, Sun JDK 1.6.0_13 on Cygwin/WinVista):

        BUILD FAILED
        C:\cygwin\home\us\svn\lucene\dev\trunk\lucene\build.xml:396: The following error occurred while executing this line:
        C:\cygwin\home\us\svn\lucene\dev\trunk\lucene\common-build.xml:324: Error deploying artifact 'org.apache.lucene:lucene-core:jar': Error deploying artifact: 
        File C:\cygwin\home\us\svn\lucene\dev\trunk\lucene\build\${project.artifactId}-${project.version}.jar does not exist
        

        Looks like some kind of failed interpolation maybe?

        I'm going to try under Linux later today.

        Show
        Steve Rowe added a comment - Hi Drew, I tried running ant generate-maven-artifacts without your patch, and I see the same failure as you do. (Actually, when I first tried, with Ant 1.8.1, it wasn't able to find the maven ant tasks jar (the most recent) that I had placed in $ANT_HOME/lib/ - I downgraded to Ant 1.7.0 and everything went fine. Apparently Ant 1.8.X no longer pays attention to jars in $ANT_HOME/lib/ ?) However, after applying your patch, the build still fails on trunk (haven't tried branch_3x yet), though not with the same failure. (Ant 1.7.0, Sun JDK 1.6.0_13 on Cygwin/WinVista): BUILD FAILED C:\cygwin\home\us\svn\lucene\dev\trunk\lucene\build.xml:396: The following error occurred while executing this line: C:\cygwin\home\us\svn\lucene\dev\trunk\lucene\common-build.xml:324: Error deploying artifact 'org.apache.lucene:lucene-core:jar': Error deploying artifact: File C:\cygwin\home\us\svn\lucene\dev\trunk\lucene\build\${project.artifactId}-${project.version}.jar does not exist Looks like some kind of failed interpolation maybe? I'm going to try under Linux later today.
        Hide
        Uwe Schindler added a comment - - edited

        Hi,

        • ANT 1.8 is not recommeneded for Lucene builds, some specials in Lucene do not work correctly, because some internal changes to handling of sysprops and other is not working correcty. Use Ant 1.7.x
        • Both Lucene 3.x and trunk build correctly without your patch, as it can be seen every night on Hudson.
        • There are some warnings and errors about missing artifacts, but the JARs are built correctly - so not fatal - that was always the case and we dont want to touch this
        • Building maven artifacts from inside contrib dirs do not work, only from top-level lucene subdir
        • Building maven artifacts from the lucene/solr main folder does not work, too (I mean directly below trunk or branch_3x) -> be sure to be inside lucene folder

        See hudson builds:

        Maybe your maven.jar version is wrong, we are using 2.0.9 on hudson. This version is used since ages to also build the release artifacts (see release todo in wiki):

        [hudson@lucene ~]$ ls -l ~/.ant/lib/
        total 1831
        -rw-r--r--  1 hudson  hudson   701049 Jul 27  2006 js.jar
        -rw-r--r--  1 hudson  hudson  1018394 Sep 30 20:28 maven-ant-tasks-2.0.9.jar
        -rw-r--r--  1 hudson  hudson    34607 Oct 16  2006 script-js.jar
        

        (the js.jar and script.js.jar are in our ant lib folder as we are using OpenJDK on FreeBSD and that has no JavaScript for testing Solr)

        Show
        Uwe Schindler added a comment - - edited Hi, ANT 1.8 is not recommeneded for Lucene builds, some specials in Lucene do not work correctly, because some internal changes to handling of sysprops and other is not working correcty. Use Ant 1.7.x Both Lucene 3.x and trunk build correctly without your patch, as it can be seen every night on Hudson. There are some warnings and errors about missing artifacts, but the JARs are built correctly - so not fatal - that was always the case and we dont want to touch this Building maven artifacts from inside contrib dirs do not work, only from top-level lucene subdir Building maven artifacts from the lucene/solr main folder does not work, too (I mean directly below trunk or branch_3x) -> be sure to be inside lucene folder See hudson builds: https://hudson.apache.org/hudson/job/Lucene-trunk/1369/consoleText https://hudson.apache.org/hudson/job/Lucene-3.x/187/consoleText Maybe your maven.jar version is wrong, we are using 2.0.9 on hudson. This version is used since ages to also build the release artifacts (see release todo in wiki): [hudson@lucene ~]$ ls -l ~/.ant/lib/ total 1831 -rw-r--r-- 1 hudson hudson 701049 Jul 27 2006 js.jar -rw-r--r-- 1 hudson hudson 1018394 Sep 30 20:28 maven-ant-tasks-2.0.9.jar -rw-r--r-- 1 hudson hudson 34607 Oct 16 2006 script-js.jar (the js.jar and script.js.jar are in our ant lib folder as we are using OpenJDK on FreeBSD and that has no JavaScript for testing Solr)
        Hide
        Drew Farris added a comment -

        Hi Steve, Uwe,

        I was using Ant 1.7.1 with 1.6.0_22, maven-ant-tasks 2.2.1 on ubuntu 10.04 – after switching to maven-ant-tasks 2.0.9 the builds for branch 3.x and trunk work fine without the patch I proposed.

        FWIW, I was able to fetch 2.0.9 from the archive: http://archive.apache.org/dist/maven/binaries/maven-ant-tasks-2.0.9.jar

        Thanks for the detailed explanation Uwe.

        Show
        Drew Farris added a comment - Hi Steve, Uwe, I was using Ant 1.7.1 with 1.6.0_22, maven-ant-tasks 2.2.1 on ubuntu 10.04 – after switching to maven-ant-tasks 2.0.9 the builds for branch 3.x and trunk work fine without the patch I proposed. FWIW, I was able to fetch 2.0.9 from the archive: http://archive.apache.org/dist/maven/binaries/maven-ant-tasks-2.0.9.jar Thanks for the detailed explanation Uwe.
        Hide
        Steve Rowe added a comment -

        after switching to maven-ant-tasks 2.0.9 the builds for branch 3.x and trunk work fine without the patch I proposed.

        After switching to Ant 1.7.1 and maven-ant-tasks 2.0.9, both trunk and branch_3x now work for me too under Cygwin on Win7 without the proposed patch. (I put maven-ant-tasks-2.0.9.jar in C:\Users\sarowe\.ant\lib, rather than the Cygwin-ish /home/sarowe/.ant/lib, because Ant is running on the native Windows binary 1.6.0_21 JRE).

        Show
        Steve Rowe added a comment - after switching to maven-ant-tasks 2.0.9 the builds for branch 3.x and trunk work fine without the patch I proposed. After switching to Ant 1.7.1 and maven-ant-tasks 2.0.9, both trunk and branch_3x now work for me too under Cygwin on Win7 without the proposed patch. (I put maven-ant-tasks-2.0.9.jar in C:\Users\sarowe\.ant\lib , rather than the Cygwin-ish /home/sarowe/.ant/lib , because Ant is running on the native Windows binary 1.6.0_21 JRE).
        Hide
        Ian Boston added a comment -

        I am sure you know this, but, this patch fixes an otherwise broken build of Solr and Lucene at r1040465 using

        Apache Ant version 1.8.1 compiled on September 21 2010
        Apache Maven 2.2.1 (r801777; 2009-08-06 20:16:01+0100)
        Java version: 1.6.0_22
        OS name: "mac os x" version: "10.6.5" arch: "x86_64" Family: "mac"
        /usr/share/ant/lib/maven-ant-tasks-2.1.1.jar

        Since 1.8.1 came with the last JDK install, others might be having the same problem.
        HTH

        Show
        Ian Boston added a comment - I am sure you know this, but, this patch fixes an otherwise broken build of Solr and Lucene at r1040465 using Apache Ant version 1.8.1 compiled on September 21 2010 Apache Maven 2.2.1 (r801777; 2009-08-06 20:16:01+0100) Java version: 1.6.0_22 OS name: "mac os x" version: "10.6.5" arch: "x86_64" Family: "mac" /usr/share/ant/lib/maven-ant-tasks-2.1.1.jar Since 1.8.1 came with the last JDK install, others might be having the same problem. HTH
        Hide
        Robert Muir added a comment -

        i know, i'm re-opening a maven issue!

        But I'm working on fixes to resolve our build so it works fine on 1.7 and 1.8.

        I tend to agree with Ian's comment that since 1.8 is installed on many systems by default,
        we should just try to make things work with it nicely.

        So, if someone wants to make a patch that fixes this issue so it works on both 1.7 and 1.8,
        I think it would be a good thing. Its unclear from the comments whether this patch does
        that (it might, but i don't know, since we have no way of verifying that maven is working).

        Show
        Robert Muir added a comment - i know, i'm re-opening a maven issue! But I'm working on fixes to resolve our build so it works fine on 1.7 and 1.8. I tend to agree with Ian's comment that since 1.8 is installed on many systems by default, we should just try to make things work with it nicely. So, if someone wants to make a patch that fixes this issue so it works on both 1.7 and 1.8, I think it would be a good thing. Its unclear from the comments whether this patch does that (it might, but i don't know, since we have no way of verifying that maven is working).
        Hide
        Steve Rowe added a comment -

        I'll work on this once I have committed the patches on LUCENE-2657. For that work, I've been using Ant 1.7.1 and Maven Ant Tasks 2.0.9 without any problems. But we should make sure that Ant 1.8.X and Maven Ant Tasks 2.0.10 and 2.1.X also work.

        Show
        Steve Rowe added a comment - I'll work on this once I have committed the patches on LUCENE-2657 . For that work, I've been using Ant 1.7.1 and Maven Ant Tasks 2.0.9 without any problems. But we should make sure that Ant 1.8.X and Maven Ant Tasks 2.0.10 and 2.1.X also work.
        Hide
        Steve Rowe added a comment -

        I tested ant clean generate-maven-artifacts on branch_3x with maven-ant-tasks-2.1.1.jar and both Ant 1.7.1 and 1.8.1. Everything works.

        I'll test more combinations tomorrow.

        Show
        Steve Rowe added a comment - I tested ant clean generate-maven-artifacts on branch_3x with maven-ant-tasks-2.1.1.jar and both Ant 1.7.1 and 1.8.1. Everything works. I'll test more combinations tomorrow.
        Hide
        Steve Rowe added a comment - - edited

        Hudson nightly builds are failing when executing ant generate-maven-artifacts, with the same error as Drew reported.

        When I use maven-ant-tasks-2.0.9.jar with Ant 1.7.1 locally, and the equivalent of Drew's patches (since the POM parents have changed), I see the same errors, but the build doesn't fail.

        I just re-ran ant clean generate-maven-artifacts on trunk with an empty ~/.m2/ directory, Ant 1.7.1 and maven-ant-tasks-2.1.1.jar – no errors were printed, and the build succeeded.

        I think instead of adding stuff to the *build.xml files, we should require maven-ant-tasks-2.1.1.jar for execution of ant generate-maven-artifacts.

        Show
        Steve Rowe added a comment - - edited Hudson nightly builds are failing when executing ant generate-maven-artifacts , with the same error as Drew reported. When I use maven-ant-tasks-2.0.9.jar with Ant 1.7.1 locally, and the equivalent of Drew's patches (since the POM parents have changed), I see the same errors, but the build doesn't fail. I just re-ran ant clean generate-maven-artifacts on trunk with an empty ~/.m2/ directory, Ant 1.7.1 and maven-ant-tasks-2.1.1.jar – no errors were printed, and the build succeeded. I think instead of adding stuff to the *build.xml files, we should require maven-ant-tasks-2.1.1.jar for execution of ant generate-maven-artifacts .
        Hide
        Steve Rowe added a comment -

        It appears that there were two problems blocking the Hudson build scripts from succeeding:

        1. Maven Ant Tasks seems to dislike mixed artifact versions cached in the local maven repository (~/.m2/repository/) - this leads to errors that look like the one Drew originally reported on this issue, regardless of the version of Maven Ant Tasks that is used.
        2. maven-ant-tasks-2.0.9.jar does not work with the new full POMs introduced by LUCENE-2657 - errors that look like Drew's are printed and the build fails.

        However, purging the local maven repository (~/.m2/repository/) and upgrading to maven-ant-tasks-2.1.1.jar allows ant generate-maven-artifacts to consistently succeed.

        Uwe has changed the Hudson build setup to rm -rf $HOME/.m2/repository and has upgraded the Maven Ant Tasks jar to v2.1.1. Since these changes, the only full build that has completed is the Lucene-3.x build, and it was a success.

        I'll leave this issue open until all of the nightly Hudson builds have succeeded (hopefully that will happen tonight).

        Show
        Steve Rowe added a comment - It appears that there were two problems blocking the Hudson build scripts from succeeding: Maven Ant Tasks seems to dislike mixed artifact versions cached in the local maven repository ( ~/.m2/repository/ ) - this leads to errors that look like the one Drew originally reported on this issue, regardless of the version of Maven Ant Tasks that is used. maven-ant-tasks-2.0.9.jar does not work with the new full POMs introduced by LUCENE-2657 - errors that look like Drew's are printed and the build fails. However, purging the local maven repository ( ~/.m2/repository/ ) and upgrading to maven-ant-tasks-2.1.1.jar allows ant generate-maven-artifacts to consistently succeed. Uwe has changed the Hudson build setup to rm -rf $HOME/.m2/repository and has upgraded the Maven Ant Tasks jar to v2.1.1. Since these changes, the only full build that has completed is the Lucene-3.x build, and it was a success. I'll leave this issue open until all of the nightly Hudson builds have succeeded (hopefully that will happen tonight).
        Hide
        Steve Rowe added a comment -

        All nightlies succeeded last night - resolving.

        Show
        Steve Rowe added a comment - All nightlies succeeded last night - resolving.

          People

          • Assignee:
            Steve Rowe
            Reporter:
            Drew Farris
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development