Details

    • Type: Task Task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6, 4.0-ALPHA
    • Component/s: general/build
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      As mentioned on the ML thread: "switch jars to ivy mechanism?".

      1. LUCENE-3930.patch
        7 kB
        Robert Muir
      2. LUCENE-3930.patch
        8 kB
        Uwe Schindler
      3. LUCENE-3930.patch
        35 kB
        Robert Muir
      4. LUCENE-3930-solr-example.patch
        6 kB
        Chris Male
      5. LUCENE-3930-solr-example.patch
        6 kB
        Chris Male
      6. ant_-verbose_clean_test.out.txt
        4.11 MB
        Hoss Man
      7. patch-jetty-build.patch
        2 kB
        Chris Male
      8. noggit-commons-csv.patch
        166 kB
        Chris Male
      9. LUCENE-3930__ivy_bootstrap_target.patch
        3 kB
        Hoss Man
      10. LUCENE-3930-skip-sources-javadoc.patch
        11 kB
        Jan Høydahl
      11. LUCENE-3930_includetestlibs_excludeexamplexml.patch
        2 kB
        Hoss Man
      12. pom.xml
        2 kB
        Chris Male
      13. langdetect-1.1.jar
        1.18 MB
        Chris Male

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          Shawn: please see LUCENE-3946 and comment there about the suggested work around

          Show
          Hoss Man added a comment - Shawn: please see LUCENE-3946 and comment there about the suggested work around
          Hide
          Michael McCandless added a comment -

          Shawn, I have similar problems with the builtin ant on Fedora 13, and when I add that same echo line, I can see that the ~/.ant/lib/ivy-2.2.0.jar is on the CLASSPATH... yet it fails the ivy-availability-check.

          I never got to the bottom of it ... but installing ant myself (1.8.2) and using that version instead worked around it...

          Show
          Michael McCandless added a comment - Shawn, I have similar problems with the builtin ant on Fedora 13, and when I add that same echo line, I can see that the ~/.ant/lib/ivy-2.2.0.jar is on the CLASSPATH... yet it fails the ivy-availability-check. I never got to the bottom of it ... but installing ant myself (1.8.2) and using that version instead worked around it...
          Hide
          Robert Muir added a comment -

          Please take this to the list (This issue is resolved)... but I don't trust the
          'ants' shipped on some of these linux os's... best to download your own ant.

          Show
          Robert Muir added a comment - Please take this to the list (This issue is resolved)... but I don't trust the 'ants' shipped on some of these linux os's... best to download your own ant.
          Hide
          Shawn Heisey added a comment -

          Did you start a new terminal session after doing ant ivy-bootstrap? FWIW, I had to restart eclipse before I could continue using the eclipse/ant integration.

          I did not think of that, as I am not using Eclipse or any other IDE. This is purely commandline via ssh. I did open a new ssh session and try again, no change.

          Show
          Shawn Heisey added a comment - Did you start a new terminal session after doing ant ivy-bootstrap? FWIW, I had to restart eclipse before I could continue using the eclipse/ant integration. I did not think of that, as I am not using Eclipse or any other IDE. This is purely commandline via ssh. I did open a new ssh session and try again, no change.
          Hide
          Mark Miller added a comment -

          Did you start a new terminal session after doing ant ivy-bootstrap? FWIW, I had to restart eclipse before I could continue using the eclipse/ant integration.

          Show
          Mark Miller added a comment - Did you start a new terminal session after doing ant ivy-bootstrap? FWIW, I had to restart eclipse before I could continue using the eclipse/ant integration.
          Hide
          Shawn Heisey added a comment -

          After printing out all the echo statements under ivy-availability, it spits out this:

          ivy-fail:

          BUILD FAILED
          /index/src/trunk/build.xml:42: The following error occurred while executing this line:
          /index/src/trunk/lucene/common-build.xml:584: The following error occurred while executing this line:
          /index/src/trunk/lucene/common-build.xml:298: Ivy is not available


          By adding <echo message="$

          {java.class.path}

          "/> to the validate section of build.xml, I got it to print out the java classpath, which includes the jar downloaded by the ivy-bootstrap option:

          [echo] /usr/share/java/ant.jar:/usr/share/java/ant-launcher.jar:/usr/share/java/jaxp_parser_impl.jar:/usr/share/java/xml-commons-apis.jar:/usr/share/java/junit.jar:/usr/share/java/ant/ant-junit.jar:/usr/share/java/ant/ant-nodeps.jar:/usr/lib/jvm/java/lib/tools.jar:/home/ncindex/.ant/lib/ivy-2.2.0.jar:/usr/share/ant/lib/ant-bootstrap.jar:/usr/share/ant/lib/ant-junit.jar:/usr/share/ant/lib/ant-nodeps.jar:/usr/share/ant/lib/ant-launcher.jar:/usr/share/ant/lib/ant.jar

          Show
          Shawn Heisey added a comment - After printing out all the echo statements under ivy-availability, it spits out this: ivy-fail: BUILD FAILED /index/src/trunk/build.xml:42: The following error occurred while executing this line: /index/src/trunk/lucene/common-build.xml:584: The following error occurred while executing this line: /index/src/trunk/lucene/common-build.xml:298: Ivy is not available By adding <echo message="$ {java.class.path} "/> to the validate section of build.xml, I got it to print out the java classpath, which includes the jar downloaded by the ivy-bootstrap option: [echo] /usr/share/java/ant.jar:/usr/share/java/ant-launcher.jar:/usr/share/java/jaxp_parser_impl.jar:/usr/share/java/xml-commons-apis.jar:/usr/share/java/junit.jar:/usr/share/java/ant/ant-junit.jar:/usr/share/java/ant/ant-nodeps.jar:/usr/lib/jvm/java/lib/tools.jar:/home/ncindex/.ant/lib/ivy-2.2.0.jar:/usr/share/ant/lib/ant-bootstrap.jar:/usr/share/ant/lib/ant-junit.jar:/usr/share/ant/lib/ant-nodeps.jar:/usr/share/ant/lib/ant-launcher.jar:/usr/share/ant/lib/ant.jar
          Hide
          Shawn Heisey added a comment -

          I can't get either branch_3x or trunk to build now, on a system that used to build branch_3x without complaint. It says that ivy is not available, even after doing "ant ivy-bootstrap" to download ivy into the home directory. Specifically I am trying to build solrj from trunk, but I can't even get "ant" in the root directory of the checkout to work. I'm on CentOS 6 with oracle jdk7 built using the city-fan.org SRPMs. Ant (1.7.1) and junit are installed from package repositories. Building a checkout of lucene_solr_3_5 on the same machine works fine.

          Show
          Shawn Heisey added a comment - I can't get either branch_3x or trunk to build now, on a system that used to build branch_3x without complaint. It says that ivy is not available, even after doing "ant ivy-bootstrap" to download ivy into the home directory. Specifically I am trying to build solrj from trunk, but I can't even get "ant" in the root directory of the checkout to work. I'm on CentOS 6 with oracle jdk7 built using the city-fan.org SRPMs. Ant (1.7.1) and junit are installed from package repositories. Building a checkout of lucene_solr_3_5 on the same machine works fine.
          Hide
          Chris Male added a comment -

          JAR which I'll be submitting to Sonatype.

          Show
          Chris Male added a comment - JAR which I'll be submitting to Sonatype.
          Hide
          Robert Muir added a comment -

          Thanks everyone!

          Show
          Robert Muir added a comment - Thanks everyone!
          Hide
          Robert Muir added a comment -

          I merged back to branch_3x. will let hudson chomp on it for a while, then mark this issue resolved
          if there are no immediate problems.

          Any later improvements can be followup issues.

          Show
          Robert Muir added a comment - I merged back to branch_3x. will let hudson chomp on it for a while, then mark this issue resolved if there are no immediate problems. Any later improvements can be followup issues.
          Hide
          Robert Muir added a comment -

          patch fixing the first two problems i mentioned above:

          Thanks Hossman: I committed this (I am looking at the 3.x merge, i don't want to find/fix the bug twice)!

          Show
          Robert Muir added a comment - patch fixing the first two problems i mentioned above: Thanks Hossman: I committed this (I am looking at the 3.x merge, i don't want to find/fix the bug twice)!
          Hide
          Chris Male added a comment - - edited

          Thanks for the review.

          1) sourceDirectory and testSourceDirectory look like default values anyway?

          Err.. yes, I used the pom the maintainer had started and forgot to pull those out. Will do.

          2) there is a newer version of jsonic in maven repositories; don't know if this matters at all.

          I will stick with the version used in the project.

          Show
          Chris Male added a comment - - edited Thanks for the review. 1) sourceDirectory and testSourceDirectory look like default values anyway? Err.. yes, I used the pom the maintainer had started and forgot to pull those out. Will do. 2) there is a newer version of jsonic in maven repositories; don't know if this matters at all. I will stick with the version used in the project.
          Hide
          Dawid Weiss added a comment -

          Looks good to me, Chris. Two minor things:
          1) sourceDirectory and testSourceDirectory look like default values anyway?
          2) there is a newer version of jsonic in maven repositories; don't know if this matters at all.

          Show
          Dawid Weiss added a comment - Looks good to me, Chris. Two minor things: 1) sourceDirectory and testSourceDirectory look like default values anyway? 2) there is a newer version of jsonic in maven repositories; don't know if this matters at all.
          Hide
          Chris Male added a comment -

          Haven't heard anything back from the language-detection project, so I'm pressing ahead. Attached the pom.xml I've created for the project, for review from anybody who knows this kinda thing.

          Show
          Chris Male added a comment - Haven't heard anything back from the language-detection project, so I'm pressing ahead. Attached the pom.xml I've created for the project, for review from anybody who knows this kinda thing.
          Hide
          Robert Muir added a comment -

          Hi guys, thanks for testing!

          Can we open separate issues for these remaining things? These problems are unrelated in
          any way to any changes in this issue and are pre-existing conditions:

          • LUCENE-2999: lucene source release does not work because packaging is fucked
          • SOLR-2435: duplicated icu4j jars in binary releases

          Its a good thing that packaging is screwed up exactly like it was before this change. Because
          i didnt really have to change its logic..., i kept what was there before.

          Thanks;

          Show
          Robert Muir added a comment - Hi guys, thanks for testing! Can we open separate issues for these remaining things? These problems are unrelated in any way to any changes in this issue and are pre-existing conditions: LUCENE-2999 : lucene source release does not work because packaging is fucked SOLR-2435 : duplicated icu4j jars in binary releases Its a good thing that packaging is screwed up exactly like it was before this change. Because i didnt really have to change its logic..., i kept what was there before. Thanks;
          Hide
          Jan Høydahl added a comment -

          We have a 7Mb jar which is included in the binary distro twice. Any way to get rid of one?

          ./contrib/analysis-extras/lib/icu4j-4.8.1.1.jar
          ./contrib/extraction/lib/icu4j-4.8.1.1.jar
          

          Also, from what I can see, solr/contrib/extraction/lib/xml-apis-1.0.b2.jar dependency is redundant - tests pass without it
          See https://issues.apache.org/jira/browse/TIKA-412 and https://issues.apache.org/jira/browse/LUCENE-2961

          Show
          Jan Høydahl added a comment - We have a 7Mb jar which is included in the binary distro twice. Any way to get rid of one? ./contrib/analysis-extras/lib/icu4j-4.8.1.1.jar ./contrib/extraction/lib/icu4j-4.8.1.1.jar Also, from what I can see, solr/contrib/extraction/lib/xml-apis-1.0.b2.jar dependency is redundant - tests pass without it See https://issues.apache.org/jira/browse/TIKA-412 and https://issues.apache.org/jira/browse/LUCENE-2961
          Hide
          Hoss Man added a comment -

          patch fixing the first two problems i mentioned above:

          • categorically exclude build.xml and ivy.xml files from solr binary packages (to prevent the ones under example from being included)
          • add parity to what files under test-framework get included in line with how contrib is treated (new patterns try to match some things that don't existing in test-framework, but i don't think that's bad – future proofs us)
          Show
          Hoss Man added a comment - patch fixing the first two problems i mentioned above: categorically exclude build.xml and ivy.xml files from solr binary packages (to prevent the ones under example from being included) add parity to what files under test-framework get included in line with how contrib is treated (new patterns try to match some things that don't existing in test-framework, but i don't think that's bad – future proofs us)
          Hide
          Hoss Man added a comment -

          I did some testing of the "packages" built using trunk (circa r1307608)...

          • we don't ship solr's build.xml (or any of the sub-build.xml files) in the "binary" artifacts, and with these changes most of the new ivy.xml files are also excluded – but for some reason these newly added files are showing up, we should probably figure out why and exclude them as well since they aren't usable and could easily people...
            • ./example/example-DIH/ivy.xml
            • ./example/example-DIH/build.xml
            • ./example/ivy.xml
            • ./example/build.xml
          • the lib's for test-framework (ant, ant-junit, and junit) aren't being included in the lucene "binary" artifacts ... for the ant jars this might (test-framework doesn't actually have any run-time deps on anything in ant does it?) but it seems like hte junit jar should be included since including lucene-test-framework.jar in your classpath is useless w/o also including junit
          • "ant ivy-bootstrap" followed by "ant test" using the lucene "source" package (lucene-4.0-SNAPSHOT-src.tgz) produces a build failure – but this may have been a problem even before ivy (note the working dir and the final error)...
          hossman@bester:~/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT$ ant test
          ...
              [junit] Testsuite: org.apache.lucene.util.junitcompat.TestReproduceMessage
              [junit] Tests run: 12, Failures: 0, Errors: 0, Time elapsed: 0.114 sec
              [junit] 
          
          test:
          
          compile-lucene-core:
          
          jflex-uptodate-check:
          
          jflex-notice:
          
          javacc-uptodate-check:
          
          javacc-notice:
          
          ivy-availability-check:
          
          ivy-fail:
          
          resolve:
          [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
          
          init:
          
          clover.setup:
          
          clover.info:
               [echo] 
               [echo]       Clover not found. Code coverage reports disabled.
               [echo]   	
          
          clover:
          
          common.compile-core:
              [javac] Compiling 1 source file to /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build/core/classes/java
          
          compile-core:
          
          compile-test-framework:
          
          ivy-availability-check:
          
          ivy-fail:
          
          resolve:
          [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
          
          init:
          
          compile-lucene-core:
          
          compile-core:
          
          compile-test:
               [echo] Building demo...
          
          ivy-availability-check:
          
          ivy-fail:
          
          resolve:
          [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
          
          common.init:
          
          compile-lucene-core:
          
          contrib-build.init:
          
          check-lucene-core-uptodate:
          
          jar-lucene-core:
          
          jflex-uptodate-check:
          
          jflex-notice:
          
          javacc-uptodate-check:
          
          javacc-notice:
          
          ivy-availability-check:
          
          ivy-fail:
          
          resolve:
          [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
          
          init:
          
          clover.setup:
          
          clover.info:
               [echo] 
               [echo]       Clover not found. Code coverage reports disabled.
               [echo]   	
          
          clover:
          
          common.compile-core:
              [javac] Compiling 1 source file to /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build/core/classes/java
          
          compile-core:
          
          jar-core:
                [jar] Building jar: /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build/core/lucene-core-4.0-SNAPSHOT.jar
          
          init:
          
          compile-test:
               [echo] Building demo...
          
          check-analyzers-common-uptodate:
          
          jar-analyzers-common:
          
          BUILD FAILED
          /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build.xml:487: The following error occurred while executing this line:
          /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/common-build.xml:1026: The following error occurred while executing this line:
          /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/contrib/contrib-build.xml:58: The following error occurred while executing this line:
          /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/common-build.xml:551: Basedir /home/hossman/tmp/ivy-pck-testing/lu/src/modules/analysis/common does not exist
          
          Total time: 5 minutes 10 seconds
          

          ...it's trying to reach back up out of the working directory into "../modules"

          Show
          Hoss Man added a comment - I did some testing of the "packages" built using trunk (circa r1307608)... we don't ship solr's build.xml (or any of the sub-build.xml files) in the "binary" artifacts, and with these changes most of the new ivy.xml files are also excluded – but for some reason these newly added files are showing up, we should probably figure out why and exclude them as well since they aren't usable and could easily people... ./example/example-DIH/ivy.xml ./example/example-DIH/build.xml ./example/ivy.xml ./example/build.xml the lib's for test-framework (ant, ant-junit, and junit) aren't being included in the lucene "binary" artifacts ... for the ant jars this might (test-framework doesn't actually have any run-time deps on anything in ant does it?) but it seems like hte junit jar should be included since including lucene-test-framework.jar in your classpath is useless w/o also including junit "ant ivy-bootstrap" followed by "ant test" using the lucene "source" package (lucene-4.0-SNAPSHOT-src.tgz) produces a build failure – but this may have been a problem even before ivy (note the working dir and the final error)... hossman@bester:~/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT$ ant test ... [junit] Testsuite: org.apache.lucene.util.junitcompat.TestReproduceMessage [junit] Tests run: 12, Failures: 0, Errors: 0, Time elapsed: 0.114 sec [junit] test: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: ivy-availability-check: ivy-fail: resolve: [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [javac] Compiling 1 source file to /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build/core/classes/java compile-core: compile-test-framework: ivy-availability-check: ivy-fail: resolve: [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml init: compile-lucene-core: compile-core: compile-test: [echo] Building demo... ivy-availability-check: ivy-fail: resolve: [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml common.init: compile-lucene-core: contrib-build.init: check-lucene-core-uptodate: jar-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: ivy-availability-check: ivy-fail: resolve: [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/.ant/lib/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [javac] Compiling 1 source file to /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build/core/classes/java compile-core: jar-core: [jar] Building jar: /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build/core/lucene-core-4.0-SNAPSHOT.jar init: compile-test: [echo] Building demo... check-analyzers-common-uptodate: jar-analyzers-common: BUILD FAILED /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/build.xml:487: The following error occurred while executing this line: /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/common-build.xml:1026: The following error occurred while executing this line: /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/contrib/contrib-build.xml:58: The following error occurred while executing this line: /home/hossman/tmp/ivy-pck-testing/lu/src/lucene-4.0-SNAPSHOT/common-build.xml:551: Basedir /home/hossman/tmp/ivy-pck-testing/lu/src/modules/analysis/common does not exist Total time: 5 minutes 10 seconds ...it's trying to reach back up out of the working directory into "../modules"
          Hide
          Simon Willnauer added a comment -

          +1 thanks guys! that worked for me though!

          Show
          Simon Willnauer added a comment - +1 thanks guys! that worked for me though!
          Hide
          Robert Muir added a comment -

          One final test, i checked out on laptop (using hossmans bootstrap), and ran 'ant resolve'
          (also 'ant idea' or 'ant eclipse' call this).. and then cut internet and ran tests/built
          and ran solr example offline. So I think its ready.

          Show
          Robert Muir added a comment - One final test, i checked out on laptop (using hossmans bootstrap), and ran 'ant resolve' (also 'ant idea' or 'ant eclipse' call this).. and then cut internet and ran tests/built and ran solr example offline. So I think its ready.
          Hide
          Robert Muir added a comment -

          I'll work on merging this back. I've tried to setup jenkins to where there will be no failed builds,
          but its the kind of thing where i'll be surprised if I get away without a failure. If there are problems,
          I will fix. If the problems are serious, I will revert.

          As far as the plan for 3.x: my idea is we bake in trunk for a while (let jenkins have its way with the
          change, etc). At the same time we can look at how to backport for hopefully next week.

          I might create a branch off of 3.x (the same way as this one worked) so that its easier for people to help.
          But first lets get it in trunk and see how that goes.

          Show
          Robert Muir added a comment - I'll work on merging this back. I've tried to setup jenkins to where there will be no failed builds, but its the kind of thing where i'll be surprised if I get away without a failure. If there are problems, I will fix. If the problems are serious, I will revert. As far as the plan for 3.x: my idea is we bake in trunk for a while (let jenkins have its way with the change, etc). At the same time we can look at how to backport for hopefully next week. I might create a branch off of 3.x (the same way as this one worked) so that its easier for people to help. But first lets get it in trunk and see how that goes.
          Hide
          Uwe Schindler added a comment -

          +1 I trust you, merge the shit back. Jenkins already has IVY, only some changes needed for build scripts to cleanup correctly

          Show
          Uwe Schindler added a comment - +1 I trust you, merge the shit back. Jenkins already has IVY, only some changes needed for build scripts to cleanup correctly
          Hide
          Michael McCandless added a comment -

          In my opinion this is ready to go into trunk. I'll wait a bit for any feedback though.

          +1

          "ant test" passes, after the one-time "ant ivy-bootstrap".

          Thanks everyone!

          Show
          Michael McCandless added a comment - In my opinion this is ready to go into trunk. I'll wait a bit for any feedback though. +1 "ant test" passes, after the one-time "ant ivy-bootstrap". Thanks everyone!
          Hide
          Dawid Weiss added a comment -

          +1. Maybe it's good that this issue came out. I think it straightened a few things out.

          Show
          Dawid Weiss added a comment - +1. Maybe it's good that this issue came out. I think it straightened a few things out.
          Hide
          Robert Muir added a comment -

          Just some stats on source release sizes:

          Trunk:
          -rw-rw-r--  1 rmuir rmuir 11372783 2012-03-30 12:25 lucene-4.0-SNAPSHOT-src.tgz
          -rw-rw-r--  1 rmuir rmuir 107074928 2012-03-30 12:20 apache-solr-4.0-SNAPSHOT-src.tgz
          
          Branch:
          -rw-rw-r--  1 rmuir rmuir  8207455 2012-03-30 12:26 lucene-4.0-SNAPSHOT-src.tgz
          -rw-rw-r--  1 rmuir rmuir 30676680 2012-03-30 12:23 apache-solr-4.0-SNAPSHOT-src.tgz
          

          I think later, we should investigate pulling some of the large test data stuff,
          like the 5MB europarl-lines file via ivy, this is easy enough to do, would be
          cached locally etc, and make the source release much smaller.

          This could also make it possible to use the 'large' one easier that jenkins uses...

          But those are optimizations for later issues...

          Show
          Robert Muir added a comment - Just some stats on source release sizes: Trunk: -rw-rw-r-- 1 rmuir rmuir 11372783 2012-03-30 12:25 lucene-4.0-SNAPSHOT-src.tgz -rw-rw-r-- 1 rmuir rmuir 107074928 2012-03-30 12:20 apache-solr-4.0-SNAPSHOT-src.tgz Branch: -rw-rw-r-- 1 rmuir rmuir 8207455 2012-03-30 12:26 lucene-4.0-SNAPSHOT-src.tgz -rw-rw-r-- 1 rmuir rmuir 30676680 2012-03-30 12:23 apache-solr-4.0-SNAPSHOT-src.tgz I think later, we should investigate pulling some of the large test data stuff, like the 5MB europarl-lines file via ivy, this is easy enough to do, would be cached locally etc, and make the source release much smaller. This could also make it possible to use the 'large' one easier that jenkins uses... But those are optimizations for later issues...
          Hide
          Ryan McKinley added a comment -

          In my opinion this is ready to go into trunk. I'll wait a bit for any feedback though.

          +1 ant test worked on my first try (after ivy bootstrap)

          HUGE thanks for looking into this

          Show
          Ryan McKinley added a comment - In my opinion this is ready to go into trunk. I'll wait a bit for any feedback though. +1 ant test worked on my first try (after ivy bootstrap) HUGE thanks for looking into this
          Hide
          Robert Muir added a comment -

          i spent about an hour blowing away everything (ant clean clean-jars) and running
          various ant targets from various locations and checking stuff out.

          In my opinion this is ready to go into trunk. I'll wait a bit for any feedback though.

          Show
          Robert Muir added a comment - i spent about an hour blowing away everything (ant clean clean-jars) and running various ant targets from various locations and checking stuff out. In my opinion this is ready to go into trunk. I'll wait a bit for any feedback though.
          Hide
          Robert Muir added a comment -

          ok i downloaded intellij, and from what i can tell its working now...
          (I can run lucene and solr tests etc and it doesnt give me any errors that indicate any issues)

          i'll be testing the various ant tasks and trying to find any bugs now.

          Show
          Robert Muir added a comment - ok i downloaded intellij, and from what i can tell its working now... (I can run lucene and solr tests etc and it doesnt give me any errors that indicate any issues) i'll be testing the various ant tasks and trying to find any bugs now.
          Hide
          Robert Muir added a comment -

          I added top-level/recursive resolve tasks to resolve any deps and fixed ant eclipse to depend on this.
          Also added 'clean-jars' top-level that basically does rm */.jar
          I would add this before the hudson 'svn status' check, so if someone commits any jars to the source tree it fails.

          At this point I think everything is generally working nice in trunk (compile/test/packaging/javadocs/maven/...)

          I'll be doing testing of all the tasks from all the modules today to find any kinks, but I think we are nearly ready
          to land in trunk.

          I'll look at intellij (i already made it depend on resolve) and see if i can get it into shape, since
          e.g. junit.jar and friends are now under test-framework/lib.

          Any help testing or fixing intellij or whatever is appreciated

          Show
          Robert Muir added a comment - I added top-level/recursive resolve tasks to resolve any deps and fixed ant eclipse to depend on this. Also added 'clean-jars' top-level that basically does rm * / .jar I would add this before the hudson 'svn status' check, so if someone commits any jars to the source tree it fails. At this point I think everything is generally working nice in trunk (compile/test/packaging/javadocs/maven/...) I'll be doing testing of all the tasks from all the modules today to find any kinks, but I think we are nearly ready to land in trunk. I'll look at intellij (i already made it depend on resolve) and see if i can get it into shape, since e.g. junit.jar and friends are now under test-framework/lib. Any help testing or fixing intellij or whatever is appreciated
          Hide
          Robert Muir added a comment -

          Jan: i committed this as we are getting pretty close... Thanks!

          Show
          Robert Muir added a comment - Jan: i committed this as we are getting pretty close... Thanks!
          Hide
          Robert Muir added a comment -

          The attached patch disables fetching of source jars and javadoc jars by default, to optimize for speed and bandwidth. Without it, the build takes much longer time.

          Jan do you want to commit this? I think its a nice improvement! I can commit it for you if you don't have the time!

          Show
          Robert Muir added a comment - The attached patch disables fetching of source jars and javadoc jars by default, to optimize for speed and bandwidth. Without it, the build takes much longer time. Jan do you want to commit this? I think its a nice improvement! I can commit it for you if you don't have the time!
          Hide
          Chris Male added a comment -

          I hadn't ever heard about it to be honest. Examining discussions on the language-detection project, the maintainer expressed an inability to find time to make a Maven release, but that was about 9 months ago. I've dropped in a comment seeing if he'd be prepared to do the release, if not, we know what we have to do.

          Show
          Chris Male added a comment - I hadn't ever heard about it to be honest. Examining discussions on the language-detection project, the maintainer expressed an inability to find time to make a Maven release, but that was about 9 months ago. I've dropped in a comment seeing if he'd be prepared to do the release, if not, we know what we have to do.
          Hide
          Robert Muir added a comment -

          https://docs.sonatype.org/display/Repository/Uploading+3rd-party+Artifacts+to+The+Central+Repository

          Thanks for bringing this up dawid. Why didn't anyone bring this up before?

          Is it considered a dark, secret option that disrupts the viral natural of maven?

          I can't believe we've had maven artifacts like solr-xxx for all these 3.x releases
          when there was an official mechanism to do this: 'as long as the license permits'
          as the page says.

          Show
          Robert Muir added a comment - https://docs.sonatype.org/display/Repository/Uploading+3rd-party+Artifacts+to+The+Central+Repository Thanks for bringing this up dawid. Why didn't anyone bring this up before? Is it considered a dark, secret option that disrupts the viral natural of maven? I can't believe we've had maven artifacts like solr-xxx for all these 3.x releases when there was an official mechanism to do this: 'as long as the license permits' as the page says.
          Hide
          Dawid Weiss added a comment -

          If we go the third party route I suggest to release an artifact with a -jdk15 classifier to make it explicit it's a 1.5 build. Perhaps we can suggest to the maintainer to compile with 1.5 compatibility if this doesn't involve any source code changes?

          Show
          Dawid Weiss added a comment - If we go the third party route I suggest to release an artifact with a -jdk15 classifier to make it explicit it's a 1.5 build. Perhaps we can suggest to the maintainer to compile with 1.5 compatibility if this doesn't involve any source code changes?
          Hide
          Chris Male added a comment -

          Good point.

          Show
          Chris Male added a comment - Good point.
          Hide
          Robert Muir added a comment -

          keep in mind the solution i have only works for trunk, because they compile with java 6.
          their code compiles just fine with a java5 compiler or with java 5 options...

          from ivy perspective i can add ant logic to just check out their source and compile it ourselves.

          but just keep it in mind for maven.

          Show
          Robert Muir added a comment - keep in mind the solution i have only works for trunk, because they compile with java 6. their code compiles just fine with a java5 compiler or with java 5 options... from ivy perspective i can add ant logic to just check out their source and compile it ourselves. but just keep it in mind for maven.
          Hide
          Chris Male added a comment -

          I'm adding a message on the project to see if the maintainer's prepared to following Sonatype's official maven release process. If not, then we can go the 3rd party route.

          Show
          Chris Male added a comment - I'm adding a message on the project to see if the maintainer's prepared to following Sonatype's official maven release process. If not, then we can go the 3rd party route.
          Hide
          Robert Muir added a comment -

          hack beyond dirty is what we need.

          we can't be releasing other peoples code as solr-xxx...

          Show
          Robert Muir added a comment - hack beyond dirty is what we need. we can't be releasing other peoples code as solr-xxx...
          Hide
          Dawid Weiss added a comment -

          You can do it with Maven by specifying an optional system dependency off the project's basedir and fetching the JAR in a preliminary phase... I think. But it's a hack beyond dirty. And it doesn't make other people's lives any easier (if somebody uses your pom).

          Show
          Dawid Weiss added a comment - You can do it with Maven by specifying an optional system dependency off the project's basedir and fetching the JAR in a preliminary phase... I think. But it's a hack beyond dirty. And it doesn't make other people's lives any easier (if somebody uses your pom).
          Hide
          Chris Male added a comment -

          For our official ant build, the jar is now retrieved as an ivy dependency so we've gotten rid of the binary jar. But its done so using what I think is an ivy only mechanism of specifying a path to a jar. The language-detection project fortunately includes a jar of itself in its own git, so we simply pull that jar. But we can't do that for Maven, I think? So yes, we need a release.

          I hadn't seen the Sonatype thing before, that is certainly worth considering.

          Show
          Chris Male added a comment - For our official ant build, the jar is now retrieved as an ivy dependency so we've gotten rid of the binary jar. But its done so using what I think is an ivy only mechanism of specifying a path to a jar. The language-detection project fortunately includes a jar of itself in its own git, so we simply pull that jar. But we can't do that for Maven, I think? So yes, we need a release. I hadn't seen the Sonatype thing before, that is certainly worth considering.
          Hide
          Dawid Weiss added a comment -

          Don't we need to get rid of the binary JAR anyway? If so, the alternatives are to either put all the sources in lucene repo or push a maven release of that JAR. SonaType accepts third-party JAR pushes too – one can do it as a last resort option.

          https://docs.sonatype.org/display/Repository/Uploading+3rd-party+Artifacts+to+The+Central+Repository

          Show
          Dawid Weiss added a comment - Don't we need to get rid of the binary JAR anyway? If so, the alternatives are to either put all the sources in lucene repo or push a maven release of that JAR. SonaType accepts third-party JAR pushes too – one can do it as a last resort option. https://docs.sonatype.org/display/Repository/Uploading+3rd-party+Artifacts+to+The+Central+Repository
          Hide
          Chris Male added a comment -

          The only real remaining issue here is how to handle the langdetect jar in Solr's langid contrib. The solution in ivy works fine (specifying the location of the jar) except I don't think you can do the same thing in Maven, which stumps our Maven releases a little. For some people that isn't an issue, but I'd like to see it solved. Does anybody know of how to do something similar in Maven?

          The other alternative is to get the language-detection project to make an actual release. It is possible to simulate a repo in a googlecode project. But I'm unsure how to go about getting the project maintainer to do that?

          Show
          Chris Male added a comment - The only real remaining issue here is how to handle the langdetect jar in Solr's langid contrib. The solution in ivy works fine (specifying the location of the jar) except I don't think you can do the same thing in Maven, which stumps our Maven releases a little. For some people that isn't an issue, but I'd like to see it solved. Does anybody know of how to do something similar in Maven? The other alternative is to get the language-detection project to make an actual release. It is possible to simulate a repo in a googlecode project. But I'm unsure how to go about getting the project maintainer to do that?
          Hide
          Robert Muir added a comment -

          +1

          i nuked ~/.ivy2 completely, and ran 'ant test' + solr 'ant example' on a clean checkout. everything works.

          Show
          Robert Muir added a comment - +1 i nuked ~/.ivy2 completely, and ran 'ant test' + solr 'ant example' on a clean checkout. everything works.
          Hide
          Jan Høydahl added a comment -

          The attached patch disables fetching of source jars and javadoc jars by default, to optimize for speed and bandwidth. Without it, the build takes much longer time.

          But it is nice to have the ability to get them automatically. So the exclude is parameterized You can enable fetching by specifying a Java option (only needed once)

          ant -Dfetch.sources.javadocs compile
          
          Show
          Jan Høydahl added a comment - The attached patch disables fetching of source jars and javadoc jars by default, to optimize for speed and bandwidth. Without it, the build takes much longer time. But it is nice to have the ability to get them automatically. So the exclude is parameterized You can enable fetching by specifying a Java option (only needed once) ant -Dfetch.sources.javadocs compile
          Hide
          Chris Male added a comment -

          For solr/contrib/extraction, could not tika-parsers use transitive="true" to avoid specifying versions for all parsers?

          Having been the one to go through and add all those parsers and versions, there were many times I wished I did use transitive. But Robert is right, at this stage we get to see clearly which jars we're using. In the future I can perhaps see places in Solr for using transitive, but in Lucene I think we should just be clear what dependencies we have.

          Why do we let ivy download sources and javadoc jars? They can be excluded in ivy.xml

          I quite like this because then we have the sources and javadocs when developing. They're downloaded once per dependency, no biggie?

          Show
          Chris Male added a comment - For solr/contrib/extraction, could not tika-parsers use transitive="true" to avoid specifying versions for all parsers? Having been the one to go through and add all those parsers and versions, there were many times I wished I did use transitive. But Robert is right, at this stage we get to see clearly which jars we're using. In the future I can perhaps see places in Solr for using transitive, but in Lucene I think we should just be clear what dependencies we have. Why do we let ivy download sources and javadoc jars? They can be excluded in ivy.xml I quite like this because then we have the sources and javadocs when developing. They're downloaded once per dependency, no biggie?
          Hide
          Jan Høydahl added a comment -

          Yes we get more control of revisions etc by being explicit. I.e. commons-codec in solr/lib is 1.6 while tika-parsers 1.0 depends on commons-codec 1.5.

          Adding these tags just before </dependencies> skips source and javadoc downloads. Have not found anything "more global" than this...

          <dependencies>
            ::
            <exclude org="*" ext="*" type="source"/> 
            <exclude org="*" ext="*" type="javadoc"/> 
          </dependencies>
          
          Show
          Jan Høydahl added a comment - Yes we get more control of revisions etc by being explicit. I.e. commons-codec in solr/lib is 1.6 while tika-parsers 1.0 depends on commons-codec 1.5. Adding these tags just before </dependencies> skips source and javadoc downloads. Have not found anything "more global" than this... <dependencies> :: <exclude org= "*" ext= "*" type= "source" /> <exclude org= "*" ext= "*" type= "javadoc" /> </dependencies>
          Hide
          Robert Muir added a comment -

          For solr/contrib/extraction, could not tika-parsers use transitive="true" to avoid specifying versions for all parsers?

          Because by turning it off always, we know exactly what jars we have from the ivy.xml. I think this is the way to go.

          Why do we let ivy download sources and javadoc jars? They can be excluded in ivy.xml

          Just optimizations that havent been done yet... no reason really.

          Show
          Robert Muir added a comment - For solr/contrib/extraction, could not tika-parsers use transitive="true" to avoid specifying versions for all parsers? Because by turning it off always, we know exactly what jars we have from the ivy.xml. I think this is the way to go. Why do we let ivy download sources and javadoc jars? They can be excluded in ivy.xml Just optimizations that havent been done yet... no reason really.
          Hide
          Jan Høydahl added a comment -

          Guys, this is impressive piece of heavy committing!
          I tested the build on my MacBook. I love how the checkout is small and how it downloads everything on demand!

          First time I ran ant example the build failed with

          [ivy:retrieve] 		[FAILED     ] com.ibm.icu#icu4j;4.8.1.1!icu4j.jar: Downloaded file size doesn't match expected Content Length for http://repo1.maven.org/maven2/com/ibm/icu/icu4j/4.8.1.1/icu4j-4.8.1.1.jar. Please retry. (204243ms)
          

          However, re-running the same command then got further. But then it froze during jetty-javadoc download and I had to kill and restart. Don't know what happened, have not tried to reproduce. On third attempt all dependencies were fetched and the build succeeded.

          Questions:

          • For solr/contrib/extraction, could not tika-parsers use transitive="true" to avoid specifying versions for all parsers?
          • Why do we let ivy download sources and javadoc jars? They can be excluded in ivy.xml
          Show
          Jan Høydahl added a comment - Guys, this is impressive piece of heavy committing! I tested the build on my MacBook. I love how the checkout is small and how it downloads everything on demand! First time I ran ant example the build failed with [ivy:retrieve] [FAILED ] com.ibm.icu#icu4j;4.8.1.1!icu4j.jar: Downloaded file size doesn't match expected Content Length for http://repo1.maven.org/maven2/com/ibm/icu/icu4j/4.8.1.1/icu4j-4.8.1.1.jar. Please retry. (204243ms) However, re-running the same command then got further. But then it froze during jetty-javadoc download and I had to kill and restart. Don't know what happened, have not tried to reproduce. On third attempt all dependencies were fetched and the build succeeded. Questions: For solr/contrib/extraction, could not tika-parsers use transitive="true" to avoid specifying versions for all parsers? Why do we let ivy download sources and javadoc jars? They can be excluded in ivy.xml
          Hide
          Dawid Weiss added a comment -

          Hey Hoss, there's a typo in the target name "ivy-availablity-check".

          Show
          Dawid Weiss added a comment - Hey Hoss, there's a typo in the target name "ivy-availablity-check".
          Hide
          Robert Muir added a comment -

          +1

          Show
          Robert Muir added a comment - +1
          Hide
          Hoss Man added a comment -

          If the mountain won't come to us, we should at least make it easier for users to get to the mountain.

          attached patch fails with a helpful message if the ivy tasks can't be loaded from the ant classpath, and adds a new bootstrap target users can run to have us install ivy into ~/.ant/lib for them if they want

          I'm sure a lot of improvements could be made to the wording of the helpful message .. it's probably got some typos in it.

          hossman@bester:~/lucene/branch_lucene3930$ ant compile
          Buildfile: build.xml
          
          compile:
          
          compile-lucene-core:
          
          jflex-uptodate-check:
          
          jflex-notice:
          
          javacc-uptodate-check:
          
          javacc-notice:
          
          ivy-availablity-check:
               [echo] 
               [echo]      This build requires Ivy and Ivy could not be found in your ant classpath
               [echo] 
               [echo]      (Due to classpath issues and the recursive nature of the Lucene/Solr 
               [echo]      build system, a local copy of Ivy can not be used an loaded dynamicaly 
               [echo]      by the build.xml)
               [echo] 
               [echo]      You can either manually install a copy of Ivy 2.2.0 in your ant classpath:
               [echo]        http://ant.apache.org/manual/install.html#optionalTasks
               [echo] 
               [echo]      Or this build file can do it for you by running the Ivy Bootstrap target:
               [echo]        ant ivy-bootstrap     
               [echo]      
               [echo]      Either way you will only have to install Ivy one time.
               [echo] 
               [echo]      'ant ivy-bootstrap' will install a copy of Ivy into your Ant User Library:
               [echo]        /home/hossman/.ant/lib
               [echo]      
               [echo]      If you would prefer, you can have it installed into an alternative 
               [echo]      directory using the "-Divy_install_path=/some/path/you/choose" option, 
               [echo]      but you will have to specify this path every time you build Lucene/Solr 
               [echo]      in the future...
               [echo]        ant ivy-bootstrap -Divy_install_path=/some/path/you/choose
               [echo]        ...
               [echo]        ant -lib /some/path/you/choose clean compile
               [echo]        ...
               [echo]        ant -lib /some/path/you/choose clean compile
               [echo]     
          
          ivy-fail:
          
          BUILD FAILED
          /home/hossman/lucene/branch_lucene3930/build.xml:52: The following error occurred while executing this line:
          /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml:561: The following error occurred while executing this line:
          /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml:289: Ivy is not available
          
          Total time: 1 second
          
          Show
          Hoss Man added a comment - If the mountain won't come to us, we should at least make it easier for users to get to the mountain. attached patch fails with a helpful message if the ivy tasks can't be loaded from the ant classpath, and adds a new bootstrap target users can run to have us install ivy into ~/.ant/lib for them if they want I'm sure a lot of improvements could be made to the wording of the helpful message .. it's probably got some typos in it. hossman@bester:~/lucene/branch_lucene3930$ ant compile Buildfile: build.xml compile: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: ivy-availablity-check: [echo] [echo] This build requires Ivy and Ivy could not be found in your ant classpath [echo] [echo] (Due to classpath issues and the recursive nature of the Lucene/Solr [echo] build system, a local copy of Ivy can not be used an loaded dynamicaly [echo] by the build.xml) [echo] [echo] You can either manually install a copy of Ivy 2.2.0 in your ant classpath: [echo] http://ant.apache.org/manual/install.html#optionalTasks [echo] [echo] Or this build file can do it for you by running the Ivy Bootstrap target: [echo] ant ivy-bootstrap [echo] [echo] Either way you will only have to install Ivy one time. [echo] [echo] 'ant ivy-bootstrap' will install a copy of Ivy into your Ant User Library: [echo] /home/hossman/.ant/lib [echo] [echo] If you would prefer, you can have it installed into an alternative [echo] directory using the "-Divy_install_path=/some/path/you/choose" option, [echo] but you will have to specify this path every time you build Lucene/Solr [echo] in the future... [echo] ant ivy-bootstrap -Divy_install_path=/some/path/you/choose [echo] ... [echo] ant -lib /some/path/you/choose clean compile [echo] ... [echo] ant -lib /some/path/you/choose clean compile [echo] ivy-fail: BUILD FAILED /home/hossman/lucene/branch_lucene3930/build.xml:52: The following error occurred while executing this line: /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml:561: The following error occurred while executing this line: /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml:289: Ivy is not available Total time: 1 second
          Hide
          Mark Miller added a comment -

          Let's try and maintain some self control and avoid the personal attacks. This type of thing is damaging to the whole community and it's not okay.

          Show
          Mark Miller added a comment - Let's try and maintain some self control and avoid the personal attacks. This type of thing is damaging to the whole community and it's not okay.
          Hide
          Dawid Weiss added a comment -

          Moving C2 JARs out means we will have to re-release an archival version of C2 just for this purpose. The reasons are that we depend on libraries which themselves don't have 1.5 equivalents (mahout-math) so any newer version would have to be re-released along with backcompat of these libraries too... long story.

          Anyway, I will release a weird-looking version 3.5.0.1 which will be 1.5 compatible. I will let you know (and possibly modify the branch) once this happens. Give me a few hours, it'll require some checks/ testing.

          Show
          Dawid Weiss added a comment - Moving C2 JARs out means we will have to re-release an archival version of C2 just for this purpose. The reasons are that we depend on libraries which themselves don't have 1.5 equivalents (mahout-math) so any newer version would have to be re-released along with backcompat of these libraries too... long story. Anyway, I will release a weird-looking version 3.5.0.1 which will be 1.5 compatible. I will let you know (and possibly modify the branch) once this happens. Give me a few hours, it'll require some checks/ testing.
          Hide
          Yonik Seeley added a comment -

          Let's all please try to keep discussions civil folks.

          Show
          Yonik Seeley added a comment - Let's all please try to keep discussions civil folks.
          Hide
          Robert Muir added a comment -

          Too late for this issue, because we are doing all the work, and you are just whining.

          Meanwhile, the noggit problem you are bitching about is in fact 100% your fault. Get over your control issues and make it a real open source project with real releases already.

          Show
          Robert Muir added a comment - Too late for this issue, because we are doing all the work, and you are just whining. Meanwhile, the noggit problem you are bitching about is in fact 100% your fault. Get over your control issues and make it a real open source project with real releases already.
          Hide
          Yonik Seeley added a comment -

          if anyone else has a different opinion and is willing to sacrifice days of their life arguing with apache board members instead

          The third option was to do nothing hasty and wait for the actual board to come to a decision and communicate with PMCs. Too late for this issue... but something to consider for the future.

          Show
          Yonik Seeley added a comment - if anyone else has a different opinion and is willing to sacrifice days of their life arguing with apache board members instead The third option was to do nothing hasty and wait for the actual board to come to a decision and communicate with PMCs. Too late for this issue... but something to consider for the future.
          Hide
          Chris Male added a comment -

          I'm looking at changing how we handle commons-csv and noggit, I'll get there.

          Show
          Chris Male added a comment - I'm looking at changing how we handle commons-csv and noggit, I'll get there.
          Hide
          Dawid Weiss added a comment -

          Feels like the cure is worse than the disease here...

          I fully agree, but see discussion at the thread pointed to by Robert.

          Show
          Dawid Weiss added a comment - Feels like the cure is worse than the disease here... I fully agree, but see discussion at the thread pointed to by Robert.
          Hide
          Robert Muir added a comment -

          As i said on my email, given the discussion about this on the other thread, i'd rather fix shit than argue
          on mailing lists about whether our release is legal or not. If anyone else has a different opinion and is
          willing to sacrifice days of their life arguing with apache board members instead, then stand up and we'll
          go with whats in branch_3x now.

          Show
          Robert Muir added a comment - As i said on my email, given the discussion about this on the other thread, i'd rather fix shit than argue on mailing lists about whether our release is legal or not. If anyone else has a different opinion and is willing to sacrifice days of their life arguing with apache board members instead, then stand up and we'll go with whats in branch_3x now.
          Hide
          Yonik Seeley added a comment -

          Ugh... We copied the source for commons-csv and noggit? Feels like the cure is worse than the disease here...

          Show
          Yonik Seeley added a comment - Ugh... We copied the source for commons-csv and noggit? Feels like the cure is worse than the disease here...
          Hide
          Robert Muir added a comment -

          I think its not ideal, but it nukes the jars, so lets do it. I can't think of a solution that really is ideal there...

          Show
          Robert Muir added a comment - I think its not ideal, but it nukes the jars, so lets do it. I can't think of a solution that really is ideal there...
          Hide
          Chris Male added a comment -

          Patch which places noggit and commons csv under org.apache.solr.internal.* and updates everything.

          Will definitely need to include a package.html for the internal package detailing its role.

          Show
          Chris Male added a comment - Patch which places noggit and commons csv under org.apache.solr.internal.* and updates everything. Will definitely need to include a package.html for the internal package detailing its role.
          Hide
          Dawid Weiss added a comment -

          For git the revision md5 is unique and you can always do a checkout of a particular revision (typically using so-called detached head). This just moves you to a particular version in the revision tree.

          Show
          Dawid Weiss added a comment - For git the revision md5 is unique and you can always do a checkout of a particular revision (typically using so-called detached head). This just moves you to a particular version in the revision tree.
          Hide
          Robert Muir added a comment -

          And for reference, the reason we used the revision we had for that, is so that we
          would load the language profiles from the classpath and not the filesystem.

          In other words, we need this change: http://code.google.com/p/language-detection/issues/detail?id=24

          Show
          Robert Muir added a comment - And for reference, the reason we used the revision we had for that, is so that we would load the language profiles from the classpath and not the filesystem. In other words, we need this change: http://code.google.com/p/language-detection/issues/detail?id=24
          Hide
          Robert Muir added a comment -

          A checkout of head and upgrade to use that? or do a checkout of an older revision? We're going to have to checkout the jsonic source as well.

          I don't care as long as it works, and as long as its not 'head' but a specific revision/hashcode/whatever so its not changing...

          Show
          Robert Muir added a comment - A checkout of head and upgrade to use that? or do a checkout of an older revision? We're going to have to checkout the jsonic source as well. I don't care as long as it works, and as long as its not 'head' but a specific revision/hashcode/whatever so its not changing...
          Hide
          Chris Male added a comment -

          For the unreleased jars, can we just include their source in our source? This would work for commons-csv and noggit which are the only two in solr/lib. They're both very small jars with only a handful of java files.

          Any opinions on this? Do I need to change the packages for anything that we put into our source?

          Ouch, i didn't notice they changed over to git... The git repository they have just doesn't correspond
          to any meaningful rev numbers (die git die), but still has the history from svn. We are going to
          have to do a git checkout i think?

          A checkout of head and upgrade to use that? or do a checkout of an older revision? We're going to have to checkout the jsonic source as well.

          We likely also won't have any of the IDE configurations working unless someone has the time to jump into that, and I have no idea if the maven build will break by us not having jars in the source tree. These are all things I will ignore without hesitation because they don't need to block a release. If anyone wants to fix this stuff, just start committing to the branch, i need your help... its just that simple

          I think both IDE support and Maven will be fine. The idea and eclipse targets will need to invoke resolve to ensure the libs are downloaded. Once they're in the same place as they were before, the IDE configurations should march along fine. Any libs we've changed the name of or dropped, we can just update the configs for. For Maven all this work is actually going to simplify the poms. We just need to bring the dependencies inline with the libs we have. Both wont be big deals.

          Show
          Chris Male added a comment - For the unreleased jars, can we just include their source in our source? This would work for commons-csv and noggit which are the only two in solr/lib. They're both very small jars with only a handful of java files. Any opinions on this? Do I need to change the packages for anything that we put into our source? Ouch, i didn't notice they changed over to git... The git repository they have just doesn't correspond to any meaningful rev numbers (die git die), but still has the history from svn. We are going to have to do a git checkout i think? A checkout of head and upgrade to use that? or do a checkout of an older revision? We're going to have to checkout the jsonic source as well. We likely also won't have any of the IDE configurations working unless someone has the time to jump into that, and I have no idea if the maven build will break by us not having jars in the source tree. These are all things I will ignore without hesitation because they don't need to block a release. If anyone wants to fix this stuff, just start committing to the branch, i need your help... its just that simple I think both IDE support and Maven will be fine. The idea and eclipse targets will need to invoke resolve to ensure the libs are downloaded. Once they're in the same place as they were before, the IDE configurations should march along fine. Any libs we've changed the name of or dropped, we can just update the configs for. For Maven all this work is actually going to simplify the poms. We just need to bring the dependencies inline with the libs we have. Both wont be big deals.
          Hide
          Robert Muir added a comment -

          FWIW.... i did a completley clean checkout of the lucene3930 (r1306662) and got the following build failure trying to run "ant clean test" from the top level.

          Hoss, thank you for checking out the branch and doing tests! this is really useful, i know
          there are more nasties hanging out there...

          Show
          Robert Muir added a comment - FWIW.... i did a completley clean checkout of the lucene3930 (r1306662) and got the following build failure trying to run "ant clean test" from the top level. Hoss, thank you for checking out the branch and doing tests! this is really useful, i know there are more nasties hanging out there...
          Hide
          Dawid Weiss added a comment -

          Clear. It's a pity we have to deal with it right before the release but I understand (or rather accept) the rationale.

          Show
          Dawid Weiss added a comment - Clear. It's a pity we have to deal with it right before the release but I understand (or rather accept) the rationale.
          Hide
          Robert Muir added a comment -

          Is a requirement that the source release must "build out of the box"? Or is it about code publication only? I don't know, I'm asking. This seems like a revolutionary build change before the release

          Yeah you are telling me. trust me i'm not happy. see my note to the dev-list:

          • we have an existing stable build/packaging tasks that works, and have been iterated on for 5 3.x release already.
          • personally i've already tested the crap out of this branch before RC, its not just jenkins but i have my own (http://sierranevada.servebeer.com/) still running now. I was planning on cutting an RC yesterday before this shit
            came up.

          So i didnt just rush into this shit and make it blocker as some knee-jerk reaction. I don't feel i have any
          other choice.

          As i said on my email, given the discussion about this on the other thread, i'd rather fix shit than argue
          on mailing lists about whether our release is legal or not. If anyone else has a different opinion and is
          willing to sacrifice days of their life arguing with apache board members instead, then stand up and we'll
          go with whats in branch_3x now.

          But given the discussion, personally I'm not going to be involved in (and will vote against) any release
          that has jar files in the source.tgz.

          Show
          Robert Muir added a comment - Is a requirement that the source release must "build out of the box"? Or is it about code publication only? I don't know, I'm asking. This seems like a revolutionary build change before the release Yeah you are telling me. trust me i'm not happy. see my note to the dev-list: we have an existing stable build/packaging tasks that works, and have been iterated on for 5 3.x release already. personally i've already tested the crap out of this branch before RC, its not just jenkins but i have my own ( http://sierranevada.servebeer.com/ ) still running now. I was planning on cutting an RC yesterday before this shit came up. So i didnt just rush into this shit and make it blocker as some knee-jerk reaction. I don't feel i have any other choice. As i said on my email, given the discussion about this on the other thread, i'd rather fix shit than argue on mailing lists about whether our release is legal or not. If anyone else has a different opinion and is willing to sacrifice days of their life arguing with apache board members instead, then stand up and we'll go with whats in branch_3x now. But given the discussion, personally I'm not going to be involved in (and will vote against) any release that has jar files in the source.tgz.
          Hide
          Dawid Weiss added a comment -

          Having jars in our source release is.

          Is a requirement that the source release must "build out of the box"? Or is it about code publication only? I don't know, I'm asking. This seems like a revolutionary build change before the release

          Having the build OOM is. So I made a tradeoff.

          I understand this, but my gut feeling still says that if you need to install ivy in an ant global space you might as well set ANT_OPTS to increase permgen... The tradeoff made is one of many.

          i need your help... its just that simple

          Can't jump into it right now, sorry. I'll take a look when I get a spare cycle though. I'm not sure it can be fixed but I'll take a look.

          Show
          Dawid Weiss added a comment - Having jars in our source release is. Is a requirement that the source release must "build out of the box"? Or is it about code publication only? I don't know, I'm asking. This seems like a revolutionary build change before the release Having the build OOM is. So I made a tradeoff. I understand this, but my gut feeling still says that if you need to install ivy in an ant global space you might as well set ANT_OPTS to increase permgen... The tradeoff made is one of many. i need your help... its just that simple Can't jump into it right now, sorry. I'll take a look when I get a spare cycle though. I'm not sure it can be fixed but I'll take a look.
          Hide
          Robert Muir added a comment -

          Sure, but this introduces additional complexity in setting up shell aliases or simply remembering about it. This just feels wrong somehow (making life harder instead of simpler). I may take a look at it at some point but I also remember having difficulties with multiple antlib definitions in our stuff and I believe you it's a pain.

          Right but honestly, while it might not be ideal, I don't think its a blocker? Having jars in our source release is. Having the build OOM is. So I made a tradeoff.

          If you (or any one else) has the time to look at stuff like this, it would be really appreciated: thats why i created a branch:

          https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3930

          We likely also won't have any of the IDE configurations working unless someone has the time to jump into that, and I have no idea if the maven build will break by us not having jars in the source tree. These are all things I will ignore without hesitation because they don't need to block a release. If anyone wants to fix this stuff, just start committing to the branch, i need your help... its just that simple

          Show
          Robert Muir added a comment - Sure, but this introduces additional complexity in setting up shell aliases or simply remembering about it. This just feels wrong somehow (making life harder instead of simpler). I may take a look at it at some point but I also remember having difficulties with multiple antlib definitions in our stuff and I believe you it's a pain. Right but honestly, while it might not be ideal, I don't think its a blocker? Having jars in our source release is. Having the build OOM is. So I made a tradeoff. If you (or any one else) has the time to look at stuff like this, it would be really appreciated: thats why i created a branch: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3930 We likely also won't have any of the IDE configurations working unless someone has the time to jump into that, and I have no idea if the maven build will break by us not having jars in the source tree. These are all things I will ignore without hesitation because they don't need to block a release. If anyone wants to fix this stuff, just start committing to the branch, i need your help... its just that simple
          Hide
          Dawid Weiss added a comment -

          You can use ant -lib too if you want, nothing stops you from doing that.

          Sure, but this introduces additional complexity in setting up shell aliases or simply remembering about it. This just feels wrong somehow (making life harder instead of simpler). I may take a look at it at some point but I also remember having difficulties with multiple antlib definitions in our stuff and I believe you it's a pain.

          Show
          Dawid Weiss added a comment - You can use ant -lib too if you want, nothing stops you from doing that. Sure, but this introduces additional complexity in setting up shell aliases or simply remembering about it. This just feels wrong somehow (making life harder instead of simpler). I may take a look at it at some point but I also remember having difficulties with multiple antlib definitions in our stuff and I believe you it's a pain.
          Hide
          Robert Muir added a comment -

          Again – can we make it project-wide ($

          Unknown macro: {project.home}

          /build.properties) or at least have a unique name (~/lucene-solr.properties)?

          its always been:

            <!-- Give user a chance to override without editing this file
                (and without typing -D each time it compiles it -->
            <property file="${user.home}/lucene.build.properties"/>
            <property file="${user.home}/build.properties"/>
            <property file="${basedir}/build.properties"/>
            <property file="${common.dir}/build.properties"/>
          

          So just use lucene.build.properties if you want that.

          Show
          Robert Muir added a comment - Again – can we make it project-wide ($ Unknown macro: {project.home} /build.properties) or at least have a unique name (~/lucene-solr.properties)? its always been: <!-- Give user a chance to override without editing this file (and without typing -D each time it compiles it --> <property file="${user.home}/lucene.build.properties"/> <property file="${user.home}/build.properties"/> <property file="${basedir}/build.properties"/> <property file="${common.dir}/build.properties"/> So just use lucene.build.properties if you want that.
          Hide
          Robert Muir added a comment -

          I personally don't like it when I need to install ant dependencies in a global scope – this may not be a problem from one project's perspective but if you're working on multiple projects then this can result in global dependencies shadowing project's local definitions and debugging this is a pain.

          Not to mention it's another requirement after checkout. I won't be able to look into this now, just expressing my opinion.

          There's nothing I can do about this. I spent all day trying to look for a better solution.
          You can use ant -lib too if you want, nothing stops you from doing that.

          But we have to remove all these jars, and for the build to still work. I'm sure ill piss a lot of people off.

          Show
          Robert Muir added a comment - I personally don't like it when I need to install ant dependencies in a global scope – this may not be a problem from one project's perspective but if you're working on multiple projects then this can result in global dependencies shadowing project's local definitions and debugging this is a pain. Not to mention it's another requirement after checkout. I won't be able to look into this now, just expressing my opinion. There's nothing I can do about this. I spent all day trying to look for a better solution. You can use ant -lib too if you want, nothing stops you from doing that. But we have to remove all these jars, and for the build to still work. I'm sure ill piss a lot of people off.
          Hide
          Dawid Weiss added a comment -

          ~/build.properties

          Again – can we make it project-wide ($

          {project.home}

          /build.properties) or at least have a unique name (~/lucene-solr.properties)?

          Show
          Dawid Weiss added a comment - ~/build.properties Again – can we make it project-wide ($ {project.home} /build.properties) or at least have a unique name (~/lucene-solr.properties)?
          Hide
          Robert Muir added a comment -

          Proof of concept patch to show how we can download jetty, patch it, build it and copy the jars of interest into lib.

          Yeah, thats how we should do it. Long term, for developers (like committers) that do this every day,
          we can add a property you can define in your ~/build.properties declaring where a patched jar already exists
          so it doesn't slow us down: but thats an optimization for guys like us.

          I'm running into bigger problems with the libs in the solr contrib langid. The langdect jar is from a revision I cannot find in its googlecode project (due to a switch from svn to git) and the jsonic jar it depends on isn't available in ivy accessible repos, only version 1.2.8 is.

          Ouch, i didn't notice they changed over to git... The git repository they have just doesn't correspond
          to any meaningful rev numbers (die git die), but still has the history from svn. We are going to
          have to do a git checkout i think?

          Show
          Robert Muir added a comment - Proof of concept patch to show how we can download jetty, patch it, build it and copy the jars of interest into lib. Yeah, thats how we should do it. Long term, for developers (like committers) that do this every day, we can add a property you can define in your ~/build.properties declaring where a patched jar already exists so it doesn't slow us down: but thats an optimization for guys like us. I'm running into bigger problems with the libs in the solr contrib langid. The langdect jar is from a revision I cannot find in its googlecode project (due to a switch from svn to git) and the jsonic jar it depends on isn't available in ivy accessible repos, only version 1.2.8 is. Ouch, i didn't notice they changed over to git... The git repository they have just doesn't correspond to any meaningful rev numbers (die git die), but still has the history from svn. We are going to have to do a git checkout i think?
          Hide
          Dawid Weiss added a comment -

          So you have to install ivy in your ~/.ant/lib

          I personally don't like it when I need to install ant dependencies in a global scope – this may not be a problem from one project's perspective but if you're working on multiple projects then this can result in global dependencies shadowing project's local definitions and debugging this is a pain.

          Not to mention it's another requirement after checkout. I won't be able to look into this now, just expressing my opinion.

          Show
          Dawid Weiss added a comment - So you have to install ivy in your ~/.ant/lib I personally don't like it when I need to install ant dependencies in a global scope – this may not be a problem from one project's perspective but if you're working on multiple projects then this can result in global dependencies shadowing project's local definitions and debugging this is a pain. Not to mention it's another requirement after checkout. I won't be able to look into this now, just expressing my opinion.
          Hide
          Robert Muir added a comment -

          I read up enough on that issue to know I don't want to be screwing around with it.

          So you have to install ivy in your ~/.ant/lib: I removed any automagical shit, and this is now solved.

          Show
          Robert Muir added a comment - I read up enough on that issue to know I don't want to be screwing around with it. So you have to install ivy in your ~/.ant/lib: I removed any automagical shit, and this is now solved.
          Hide
          Robert Muir added a comment -

          as far as the PermGen OOM goes, the -verbose logs show repeated instances of "Trying to override old definition of task antlib:org.apache.ivy.ant:..." suggesting that rmuir's unless="ivy.uptodate" isn't working the way we think it should (possibly because of the way the various build files are included in one another?)

          Thats right... I don't have a good solution here yet.

          Show
          Robert Muir added a comment - as far as the PermGen OOM goes, the -verbose logs show repeated instances of "Trying to override old definition of task antlib:org.apache.ivy.ant:..." suggesting that rmuir's unless="ivy.uptodate" isn't working the way we think it should (possibly because of the way the various build files are included in one another?) Thats right... I don't have a good solution here yet.
          Hide
          Chris Male added a comment -

          Proof of concept patch to show how we can download jetty, patch it, build it and copy the jars of interest into lib.

          I've made this against the 3930 branch but its targeting 3x of course.

          Show
          Chris Male added a comment - Proof of concept patch to show how we can download jetty, patch it, build it and copy the jars of interest into lib. I've made this against the 3930 branch but its targeting 3x of course.
          Hide
          Hoss Man added a comment -

          as far as the PermGen OOM goes, the -verbose logs show repeated instances of "Trying to override old definition of task antlib:org.apache.ivy.ant:..." suggesting that rmuir's unless="ivy.uptodate" isn't working the way we think it should (possibly because of the way the various build files are included in one another?)

          If we can't keep the <taskdef/ from running multiple times, we might be able to force it to use the same class loader each time, per the ant docs...
          https://ant.apache.org/manual/Tasks/typedef.html

          If you are defining tasks or types that share the same classpath with multiple taskdef or typedef tasks, the corresponding classes will be loaded by different Java ClassLoaders. Two classes with the same name loaded via different ClassLoaders are not the same class from the point of view of the Java VM, they don't share static variables and instances of these classes can't access private methods or attributes of instances defined by "the other class" of the same name. They don't even belong to the same Java package and can't access package private code, either.

          The best way to load several tasks/types that are supposed to cooperate with each other via shared Java code is to use the resource attribute and an antlib descriptor. If this is not possible, the second best option is to use the loaderref attribute and specify the same name for each and every typedef/taskdef - this way the classes will share the same ClassLoader. Note that the typedef/taskdef tasks must use identical classpath defintions (this includes the order of path components) for the loaderref attribute to work.

          ...it appears it's just some unique string key to name the classloader? (no idea if it whatever is causing hte current problem will still plague by creating multiple loaders with the same name)...
          https://svn.apache.org/viewvc/ant/core/tags/ANT_171/src/etc/testcases/core/loaderref/

          Show
          Hoss Man added a comment - as far as the PermGen OOM goes, the -verbose logs show repeated instances of "Trying to override old definition of task antlib:org.apache.ivy.ant:..." suggesting that rmuir's unless="ivy.uptodate" isn't working the way we think it should (possibly because of the way the various build files are included in one another?) If we can't keep the <taskdef/ from running multiple times, we might be able to force it to use the same class loader each time, per the ant docs... https://ant.apache.org/manual/Tasks/typedef.html If you are defining tasks or types that share the same classpath with multiple taskdef or typedef tasks, the corresponding classes will be loaded by different Java ClassLoaders. Two classes with the same name loaded via different ClassLoaders are not the same class from the point of view of the Java VM, they don't share static variables and instances of these classes can't access private methods or attributes of instances defined by "the other class" of the same name. They don't even belong to the same Java package and can't access package private code, either. The best way to load several tasks/types that are supposed to cooperate with each other via shared Java code is to use the resource attribute and an antlib descriptor. If this is not possible, the second best option is to use the loaderref attribute and specify the same name for each and every typedef/taskdef - this way the classes will share the same ClassLoader. Note that the typedef/taskdef tasks must use identical classpath defintions (this includes the order of path components) for the loaderref attribute to work. ...it appears it's just some unique string key to name the classloader? (no idea if it whatever is causing hte current problem will still plague by creating multiple loaders with the same name)... https://svn.apache.org/viewvc/ant/core/tags/ANT_171/src/etc/testcases/core/loaderref/
          Hide
          Chris Male added a comment -

          Now we have the problematic patched jars and the renamed unreleased jars.

          For the unreleased jars, can we just include their source in our source? This would work for commons-csv and noggit which are the only two in solr/lib. They're both very small jars with only a handful of java files.

          I'm running into bigger problems with the libs in the solr contrib langid. The langdect jar is from a revision I cannot find in its googlecode project (due to a switch from svn to git) and the jsonic jar it depends on isn't available in ivy accessible repos, only version 1.2.8 is.

          Show
          Chris Male added a comment - Now we have the problematic patched jars and the renamed unreleased jars. For the unreleased jars, can we just include their source in our source? This would work for commons-csv and noggit which are the only two in solr/lib. They're both very small jars with only a handful of java files. I'm running into bigger problems with the libs in the solr contrib langid. The langdect jar is from a revision I cannot find in its googlecode project (due to a switch from svn to git) and the jsonic jar it depends on isn't available in ivy accessible repos, only version 1.2.8 is.
          Hide
          Hoss Man added a comment -

          Attaching full stderr/stdout of a (differnet "ant -verbose clean test" run that ended with another PermGen OOM...

          hossman@bester:~/lucene/branch_lucene3930$ ant -verbose clean test 2>&1 > ant_-verbose_clean_test.out.txt
          java.lang.OutOfMemoryError: PermGen space
          	at java.lang.ClassLoader.findBootstrapClass(Native Method)
          	at java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:927)
          	at java.lang.ClassLoader.loadClass(ClassLoader.java:298)
          	at java.lang.ClassLoader.loadClass(ClassLoader.java:296)
          	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
          	at java.lang.ClassLoader.loadClass(ClassLoader.java:296)
          	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
          	at org.apache.tools.ant.DefaultLogger.formatTime(DefaultLogger.java:323)
          	at org.apache.tools.ant.DefaultLogger.buildFinished(DefaultLogger.java:170)
          	at org.apache.tools.ant.Project.fireBuildFinished(Project.java:2037)
          	at org.apache.tools.ant.Main.runBuild(Main.java:778)
          	at org.apache.tools.ant.Main.startAnt(Main.java:217)
          	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
          	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
          PermGen space
          
          Show
          Hoss Man added a comment - Attaching full stderr/stdout of a (differnet "ant -verbose clean test" run that ended with another PermGen OOM... hossman@bester:~/lucene/branch_lucene3930$ ant -verbose clean test 2>&1 > ant_-verbose_clean_test.out.txt java.lang.OutOfMemoryError: PermGen space at java.lang.ClassLoader.findBootstrapClass(Native Method) at java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:927) at java.lang.ClassLoader.loadClass(ClassLoader.java:298) at java.lang.ClassLoader.loadClass(ClassLoader.java:296) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:296) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) at org.apache.tools.ant.DefaultLogger.formatTime(DefaultLogger.java:323) at org.apache.tools.ant.DefaultLogger.buildFinished(DefaultLogger.java:170) at org.apache.tools.ant.Project.fireBuildFinished(Project.java:2037) at org.apache.tools.ant.Main.runBuild(Main.java:778) at org.apache.tools.ant.Main.startAnt(Main.java:217) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104) PermGen space
          Hide
          Hoss Man added a comment -

          here's the tail end of "ant -verbose clean test"...

          hossman@bester:~/lucene/branch_lucene3930$ ant -verbose clean test
          
          ...
          
          compile-test-framework:
          Skipped because property 'lucene.test.framework.compiled' set.
          
          common.compile-test:
              [mkdir] Skipping /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test because it already exists.
          Property "run.clover" has not been set
              [javac] org/apache/lucene/demo/TestDemo.java omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/TestDemo.class is up to date.
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/apache1.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/apache1.1.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/apache2.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/cpl1.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/epl1.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/freebsd.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/gpl1.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/gpl2.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/gpl3.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/lgpl2.1.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/lgpl3.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/lpgl2.0.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mit.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mozilla1.1.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_firefox3.txt skipped - don't know how to handle it
              [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_thunderbird2.txt skipped - don't know how to handle it
               [copy] org/apache/lucene/demo/test-files/docs/apache1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/apache1.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/apache1.1.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/apache1.1.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/apache2.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/apache2.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/cpl1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/cpl1.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/epl1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/epl1.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/freebsd.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/freebsd.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/gpl1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/gpl1.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/gpl2.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/gpl2.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/gpl3.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/gpl3.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/lgpl2.1.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/lgpl2.1.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/lgpl3.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/lgpl3.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/lpgl2.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/lpgl2.0.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/mit.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mit.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/mozilla1.1.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mozilla1.1.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/mozilla_eula_firefox3.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_firefox3.txt is up to date.
               [copy] org/apache/lucene/demo/test-files/docs/mozilla_eula_thunderbird2.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_thunderbird2.txt is up to date.
               [copy]  omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test is up to date.
               [copy] org omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org is up to date.
               [copy] org/apache omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache is up to date.
               [copy] org/apache/lucene omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene is up to date.
               [copy] org/apache/lucene/demo omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo is up to date.
               [copy] org/apache/lucene/demo/test-files omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files is up to date.
               [copy] org/apache/lucene/demo/test-files/docs omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs is up to date.
            [antcall] Exiting /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/build.xml.
          
          compile-tools:
          Detected Java version: 1.6 in: /usr/lib/jvm/java-6-sun-1.6.0.24/jre
          Detected OS: Linux
          Project base dir set to: /home/hossman/lucene/branch_lucene3930/lucene/tools
                [ant] calling target(s) [compile-core] in build file /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml
          parsing buildfile /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml with URI = file:/home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml
          Project base dir set to: /home/hossman/lucene/branch_lucene3930/lucene/tools
          Importing file /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml from /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml
          parsing buildfile /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml with URI = file:/home/hossman/lucene/branch_lucene3930/lucene/common-build.xml
          Already defined in main or a previous import, ignore compile-core
          Already defined in main or a previous import, ignore javadocs
           [property] Loading /home/hossman/lucene.build.properties
           [property] Unable to find property file: /home/hossman/lucene.build.properties
           [property] Loading /home/hossman/build.properties
           [property] Unable to find property file: /home/hossman/build.properties
           [property] Loading /home/hossman/lucene/branch_lucene3930/lucene/tools/build.properties
           [property] Unable to find property file: /home/hossman/lucene/branch_lucene3930/lucene/tools/build.properties
           [property] Loading /home/hossman/lucene/branch_lucene3930/lucene/build.properties
           [property] Unable to find property file: /home/hossman/lucene/branch_lucene3930/lucene/build.properties
          Override ignored for property "DSTAMP"
          Override ignored for property "TSTAMP"
          [available] Found: /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar
          Property "run.clover" has not been set
          [available] Found: javadoc/java6/package-list
          Override ignored for property "build.dir"
          [available] Unable to load class com.cenqua.clover.tasks.CloverReportTask to set property clover.present
          Importing file /home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml from /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml
          parsing buildfile /home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml with URI = file:/home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml
           [macrodef] creating macro  license-check-macro
          Trying to override old definition of task m2-deploy
           [macrodef] creating macro  m2-deploy
           [macrodef] creating macro  m2-deploy-with-pom-template
          Trying to override old definition of task build-manifest
           [macrodef] creating macro  build-manifest
          Trying to override old definition of task jarify
           [macrodef] creating macro  jarify
           [macrodef] creating macro  module-uptodate
           [macrodef] creating macro  compile-test-macro
          Trying to override old definition of task test-macro
           [macrodef] creating macro  test-macro
          [available] Unable to find file pom.xml to set property pom.xml.present
           [macrodef] creating macro  compile
           [macrodef] creating macro  invoke-javacc
          Trying to override old definition of task invoke-javadoc
           [macrodef] creating macro  invoke-javadoc
           [macrodef] creating macro  contrib-crawl
           [macrodef] creating macro  svn-export-source
           [macrodef] creating macro  get-svn-info
           [macrodef] creating macro  make-checksums
           [macrodef] creating macro  sign-artifacts-macro
           [macrodef] creating macro  copy-to-stage-macro
                [ant] Entering /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml...
          Build sequence for target(s) `compile-core' is [download-ivy, install-ivy, resolve, init, compile-core]
          Complete build sequence is [download-ivy, install-ivy, resolve, init, compile-core, compile-tools, validate, common.init, common.download-java6-javadoc-packagelist, common.install-ivy, check-analyzers-common-uptodate, common.clover.check, common.jar-core, clover.setup, junit-sequential, clover.info, clover, common.compile-core, junit-mkdir, compile-test-framework, compile-test, check-lucene-core-uptodate, common.jflex-uptodate-check, jar-core, jar-src, javadocs, install-maven-tasks, common.dist-maven, common.junit-mkdir, common.clean, check-queries-uptodate, jar-queries, javacc-uptodate-check, common.generate-test-reports, rat-sources-typedef, rat-sources, jflex-uptodate-check, common.javadocs, jar, javacc-check, jflex-check, dist-maven, default, jflex-notice, common.jar-lucene-core, common.javacc-notice, common.compile-tools, common.jar-queries, common.jar-src, common.rat-sources, compile-lucene-core, download-java6-javadoc-packagelist, common.clover.setup, common.default, junit-parallel, test, common.clover, common.junit-sequential, jar-lucene-core, common.jflex-check, clover.check, common.check-queryparser-uptodate, common.compile-test, common.jar-analyzers-common, compile, clean, common.clover.info, common.validate, common.download-ivy, check-queryparser-uptodate, common.jar-queryparser, common.check-analyzers-common-uptodate, generate-test-reports, common.javacc-uptodate-check, common.resolve, common.javacc-check, common.install-maven-tasks, common.rat-sources-typedef, jar-queryparser, common.compile-test-framework, jar-analyzers-common, common.compile-lucene-core, common.junit-parallel, common.jar, common.jflex-notice, common.check-queries-uptodate, javacc-notice, common.compile, common.check-lucene-core-uptodate, common.test, ]
          
          download-ivy:
          Skipped because property 'ivy.available' set.
          
          install-ivy:
          parsing buildfile jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/ant/antlib.xml with URI = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/ant/antlib.xml
          Trying to override old definition of datatype antlib:org.apache.ivy.ant:settings
          Trying to override old definition of task antlib:org.apache.ivy.ant:configure
          Trying to override old definition of task antlib:org.apache.ivy.ant:resolve
          Trying to override old definition of task antlib:org.apache.ivy.ant:retrieve
          Trying to override old definition of task antlib:org.apache.ivy.ant:deliver
          Trying to override old definition of task antlib:org.apache.ivy.ant:publish
          Trying to override old definition of task antlib:org.apache.ivy.ant:extract
          Trying to override old definition of task antlib:org.apache.ivy.ant:cachepath
          Trying to override old definition of task antlib:org.apache.ivy.ant:cachefileset
          Trying to override old definition of task antlib:org.apache.ivy.ant:report
          Trying to override old definition of task antlib:org.apache.ivy.ant:repreport
          Trying to override old definition of task antlib:org.apache.ivy.ant:var
          Trying to override old definition of task antlib:org.apache.ivy.ant:check
          Trying to override old definition of task antlib:org.apache.ivy.ant:artifactproperty
          Trying to override old definition of task antlib:org.apache.ivy.ant:buildlist
          Trying to override old definition of task antlib:org.apache.ivy.ant:install
          Trying to override old definition of task antlib:org.apache.ivy.ant:convertpom
          Trying to override old definition of task antlib:org.apache.ivy.ant:makepom
          Trying to override old definition of task antlib:org.apache.ivy.ant:artifactreport
          Trying to override old definition of task antlib:org.apache.ivy.ant:info
          Trying to override old definition of task antlib:org.apache.ivy.ant:addpath
          Trying to override old definition of task antlib:org.apache.ivy.ant:listmodules
          Trying to override old definition of task antlib:org.apache.ivy.ant:findrevision
          Trying to override old definition of task antlib:org.apache.ivy.ant:buildnumber
          Trying to override old definition of task antlib:org.apache.ivy.ant:cleancache
          
          resolve:
          [ivy:retrieve] No ivy:settings found for the default reference 'ivy.instance'.  A default instance will be used
          [ivy:retrieve] Loading jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivy.properties
          [ivy:retrieve] searching settings file: trying /home/hossman/lucene/branch_lucene3930/lucene/tools/ivysettings.xml
          [ivy:retrieve] searching settings file: trying /home/hossman/lucene/branch_lucene3930/lucene/tools/ivyconf.xml
          [ivy:retrieve] searching settings file: trying ivysettings.xml
          [ivy:retrieve] searching settings file: trying ivyconf.xml
          [ivy:retrieve] no settings file found, using default...
          [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ ::
          [ivy:retrieve] jakarta commons httpclient not found: using jdk url handling
          [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
          [ivy:retrieve] no default ivy user dir defined: set to /home/hossman/.ivy2
          [ivy:retrieve] including url: jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings-public.xml
                [ant] Exiting /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml.
            [antcall] Exiting /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/build.xml.
             [subant] Exiting /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/build.xml.
             [subant] Exiting /home/hossman/lucene/branch_lucene3930/lucene/build.xml.
          java.lang.OutOfMemoryError: PermGen space
          	at org.apache.tools.ant.DefaultLogger.formatTime(DefaultLogger.java:323)
          	at org.apache.tools.ant.DefaultLogger.buildFinished(DefaultLogger.java:170)
          	at org.apache.tools.ant.Project.fireBuildFinished(Project.java:2037)
          	at org.apache.tools.ant.Main.runBuild(Main.java:778)
          	at org.apache.tools.ant.Main.startAnt(Main.java:217)
          	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
          	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
          PermGen space
          
          Show
          Hoss Man added a comment - here's the tail end of "ant -verbose clean test"... hossman@bester:~/lucene/branch_lucene3930$ ant -verbose clean test ... compile-test-framework: Skipped because property 'lucene.test.framework.compiled' set. common.compile-test: [mkdir] Skipping /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test because it already exists. Property "run.clover" has not been set [javac] org/apache/lucene/demo/TestDemo.java omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/TestDemo.class is up to date. [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/apache1.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/apache1.1.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/apache2.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/cpl1.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/epl1.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/freebsd.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/gpl1.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/gpl2.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/gpl3.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/lgpl2.1.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/lgpl3.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/lpgl2.0.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mit.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mozilla1.1.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_firefox3.txt skipped - don't know how to handle it [javac] /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/src/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_thunderbird2.txt skipped - don't know how to handle it [copy] org/apache/lucene/demo/test-files/docs/apache1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/apache1.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/apache1.1.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/apache1.1.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/apache2.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/apache2.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/cpl1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/cpl1.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/epl1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/epl1.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/freebsd.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/freebsd.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/gpl1.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/gpl1.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/gpl2.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/gpl2.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/gpl3.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/gpl3.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/lgpl2.1.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/lgpl2.1.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/lgpl3.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/lgpl3.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/lpgl2.0.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/lpgl2.0.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/mit.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mit.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/mozilla1.1.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mozilla1.1.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/mozilla_eula_firefox3.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_firefox3.txt is up to date. [copy] org/apache/lucene/demo/test-files/docs/mozilla_eula_thunderbird2.txt omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs/mozilla_eula_thunderbird2.txt is up to date. [copy] omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test is up to date. [copy] org omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org is up to date. [copy] org/apache omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache is up to date. [copy] org/apache/lucene omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene is up to date. [copy] org/apache/lucene/demo omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo is up to date. [copy] org/apache/lucene/demo/test-files omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files is up to date. [copy] org/apache/lucene/demo/test-files/docs omitted as /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/demo/classes/test/org/apache/lucene/demo/test-files/docs is up to date. [antcall] Exiting /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/build.xml. compile-tools: Detected Java version: 1.6 in: /usr/lib/jvm/java-6-sun-1.6.0.24/jre Detected OS: Linux Project base dir set to: /home/hossman/lucene/branch_lucene3930/lucene/tools [ant] calling target(s) [compile-core] in build file /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml parsing buildfile /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml with URI = file:/home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml Project base dir set to: /home/hossman/lucene/branch_lucene3930/lucene/tools Importing file /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml from /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml parsing buildfile /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml with URI = file:/home/hossman/lucene/branch_lucene3930/lucene/common-build.xml Already defined in main or a previous import, ignore compile-core Already defined in main or a previous import, ignore javadocs [property] Loading /home/hossman/lucene.build.properties [property] Unable to find property file: /home/hossman/lucene.build.properties [property] Loading /home/hossman/build.properties [property] Unable to find property file: /home/hossman/build.properties [property] Loading /home/hossman/lucene/branch_lucene3930/lucene/tools/build.properties [property] Unable to find property file: /home/hossman/lucene/branch_lucene3930/lucene/tools/build.properties [property] Loading /home/hossman/lucene/branch_lucene3930/lucene/build.properties [property] Unable to find property file: /home/hossman/lucene/branch_lucene3930/lucene/build.properties Override ignored for property "DSTAMP" Override ignored for property "TSTAMP" [available] Found: /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar Property "run.clover" has not been set [available] Found: javadoc/java6/package-list Override ignored for property "build.dir" [available] Unable to load class com.cenqua.clover.tasks.CloverReportTask to set property clover.present Importing file /home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml from /home/hossman/lucene/branch_lucene3930/lucene/common-build.xml parsing buildfile /home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml with URI = file:/home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml [macrodef] creating macro license-check-macro Trying to override old definition of task m2-deploy [macrodef] creating macro m2-deploy [macrodef] creating macro m2-deploy-with-pom-template Trying to override old definition of task build-manifest [macrodef] creating macro build-manifest Trying to override old definition of task jarify [macrodef] creating macro jarify [macrodef] creating macro module-uptodate [macrodef] creating macro compile-test-macro Trying to override old definition of task test-macro [macrodef] creating macro test-macro [available] Unable to find file pom.xml to set property pom.xml.present [macrodef] creating macro compile [macrodef] creating macro invoke-javacc Trying to override old definition of task invoke-javadoc [macrodef] creating macro invoke-javadoc [macrodef] creating macro contrib-crawl [macrodef] creating macro svn-export-source [macrodef] creating macro get-svn-info [macrodef] creating macro make-checksums [macrodef] creating macro sign-artifacts-macro [macrodef] creating macro copy-to-stage-macro [ant] Entering /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml... Build sequence for target(s) `compile-core' is [download-ivy, install-ivy, resolve, init, compile-core] Complete build sequence is [download-ivy, install-ivy, resolve, init, compile-core, compile-tools, validate, common.init, common.download-java6-javadoc-packagelist, common.install-ivy, check-analyzers-common-uptodate, common.clover.check, common.jar-core, clover.setup, junit-sequential, clover.info, clover, common.compile-core, junit-mkdir, compile-test-framework, compile-test, check-lucene-core-uptodate, common.jflex-uptodate-check, jar-core, jar-src, javadocs, install-maven-tasks, common.dist-maven, common.junit-mkdir, common.clean, check-queries-uptodate, jar-queries, javacc-uptodate-check, common.generate-test-reports, rat-sources-typedef, rat-sources, jflex-uptodate-check, common.javadocs, jar, javacc-check, jflex-check, dist-maven, default, jflex-notice, common.jar-lucene-core, common.javacc-notice, common.compile-tools, common.jar-queries, common.jar-src, common.rat-sources, compile-lucene-core, download-java6-javadoc-packagelist, common.clover.setup, common.default, junit-parallel, test, common.clover, common.junit-sequential, jar-lucene-core, common.jflex-check, clover.check, common.check-queryparser-uptodate, common.compile-test, common.jar-analyzers-common, compile, clean, common.clover.info, common.validate, common.download-ivy, check-queryparser-uptodate, common.jar-queryparser, common.check-analyzers-common-uptodate, generate-test-reports, common.javacc-uptodate-check, common.resolve, common.javacc-check, common.install-maven-tasks, common.rat-sources-typedef, jar-queryparser, common.compile-test-framework, jar-analyzers-common, common.compile-lucene-core, common.junit-parallel, common.jar, common.jflex-notice, common.check-queries-uptodate, javacc-notice, common.compile, common.check-lucene-core-uptodate, common.test, ] download-ivy: Skipped because property 'ivy.available' set. install-ivy: parsing buildfile jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/ant/antlib.xml with URI = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/ant/antlib.xml Trying to override old definition of datatype antlib:org.apache.ivy.ant:settings Trying to override old definition of task antlib:org.apache.ivy.ant:configure Trying to override old definition of task antlib:org.apache.ivy.ant:resolve Trying to override old definition of task antlib:org.apache.ivy.ant:retrieve Trying to override old definition of task antlib:org.apache.ivy.ant:deliver Trying to override old definition of task antlib:org.apache.ivy.ant:publish Trying to override old definition of task antlib:org.apache.ivy.ant:extract Trying to override old definition of task antlib:org.apache.ivy.ant:cachepath Trying to override old definition of task antlib:org.apache.ivy.ant:cachefileset Trying to override old definition of task antlib:org.apache.ivy.ant:report Trying to override old definition of task antlib:org.apache.ivy.ant:repreport Trying to override old definition of task antlib:org.apache.ivy.ant:var Trying to override old definition of task antlib:org.apache.ivy.ant:check Trying to override old definition of task antlib:org.apache.ivy.ant:artifactproperty Trying to override old definition of task antlib:org.apache.ivy.ant:buildlist Trying to override old definition of task antlib:org.apache.ivy.ant:install Trying to override old definition of task antlib:org.apache.ivy.ant:convertpom Trying to override old definition of task antlib:org.apache.ivy.ant:makepom Trying to override old definition of task antlib:org.apache.ivy.ant:artifactreport Trying to override old definition of task antlib:org.apache.ivy.ant:info Trying to override old definition of task antlib:org.apache.ivy.ant:addpath Trying to override old definition of task antlib:org.apache.ivy.ant:listmodules Trying to override old definition of task antlib:org.apache.ivy.ant:findrevision Trying to override old definition of task antlib:org.apache.ivy.ant:buildnumber Trying to override old definition of task antlib:org.apache.ivy.ant:cleancache resolve: [ivy:retrieve] No ivy:settings found for the default reference 'ivy.instance'. A default instance will be used [ivy:retrieve] Loading jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivy.properties [ivy:retrieve] searching settings file: trying /home/hossman/lucene/branch_lucene3930/lucene/tools/ivysettings.xml [ivy:retrieve] searching settings file: trying /home/hossman/lucene/branch_lucene3930/lucene/tools/ivyconf.xml [ivy:retrieve] searching settings file: trying ivysettings.xml [ivy:retrieve] searching settings file: trying ivyconf.xml [ivy:retrieve] no settings file found, using default... [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [ivy:retrieve] jakarta commons httpclient not found: using jdk url handling [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml [ivy:retrieve] no default ivy user dir defined: set to /home/hossman/.ivy2 [ivy:retrieve] including url: jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings-public.xml [ant] Exiting /home/hossman/lucene/branch_lucene3930/lucene/tools/build.xml. [antcall] Exiting /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/build.xml. [subant] Exiting /home/hossman/lucene/branch_lucene3930/lucene/contrib/demo/build.xml. [subant] Exiting /home/hossman/lucene/branch_lucene3930/lucene/build.xml. java.lang.OutOfMemoryError: PermGen space at org.apache.tools.ant.DefaultLogger.formatTime(DefaultLogger.java:323) at org.apache.tools.ant.DefaultLogger.buildFinished(DefaultLogger.java:170) at org.apache.tools.ant.Project.fireBuildFinished(Project.java:2037) at org.apache.tools.ant.Main.runBuild(Main.java:778) at org.apache.tools.ant.Main.startAnt(Main.java:217) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104) PermGen space
          Hide
          Hoss Man added a comment -

          I did a touch /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-FAKE.txt to work around the license checker and this time i wound up with an OOM...

          hossman@bester:~/lucene/branch_lucene3930$ ant clean test
          ....
          common.compile-test:
              [mkdir] Created dir: /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/sandbox/classes/test
              [javac] Compiling 6 source files to /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/sandbox/classes/test
              [javac] Note: Some input files use or override a deprecated API.
              [javac] Note: Recompile with -Xlint:deprecation for details.
               [copy] Copied 1 empty directory to 1 empty directory under /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/sandbox/classes/test
          
          test-contrib:
               [echo] Building demo...
          
          download-ivy:
          
          install-ivy:
          
          resolve:
          [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ ::
          [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
          java.lang.OutOfMemoryError: PermGen space
          java.lang.OutOfMemoryError: PermGen space
          	at java.lang.Throwable.getStackTraceElement(Native Method)
          	at java.lang.Throwable.getOurStackTrace(Throwable.java:591)
          	at java.lang.Throwable.printStackTrace(Throwable.java:462)
          	at java.lang.Throwable.printStackTrace(Throwable.java:451)
          	at org.apache.tools.ant.Main.startAnt(Main.java:230)
          	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257)
          	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104)
          

          running with "-verbose" to see if i can get more details on exactly where/why the OOM is happening

          Show
          Hoss Man added a comment - I did a touch /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-FAKE.txt to work around the license checker and this time i wound up with an OOM... hossman@bester:~/lucene/branch_lucene3930$ ant clean test .... common.compile-test: [mkdir] Created dir: /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/sandbox/classes/test [javac] Compiling 6 source files to /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/sandbox/classes/test [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [copy] Copied 1 empty directory to 1 empty directory under /home/hossman/lucene/branch_lucene3930/lucene/build/contrib/sandbox/classes/test test-contrib: [echo] Building demo... download-ivy: install-ivy: resolve: [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml java.lang.OutOfMemoryError: PermGen space java.lang.OutOfMemoryError: PermGen space at java.lang.Throwable.getStackTraceElement(Native Method) at java.lang.Throwable.getOurStackTrace(Throwable.java:591) at java.lang.Throwable.printStackTrace(Throwable.java:462) at java.lang.Throwable.printStackTrace(Throwable.java:451) at org.apache.tools.ant.Main.startAnt(Main.java:230) at org.apache.tools.ant.launch.Launcher.run(Launcher.java:257) at org.apache.tools.ant.launch.Launcher.main(Launcher.java:104) running with "-verbose" to see if i can get more details on exactly where/why the OOM is happening
          Hide
          Chris Male added a comment -

          Thanks hoss, I will add in a LICENSE and NOTICE for ivy.

          Show
          Chris Male added a comment - Thanks hoss, I will add in a LICENSE and NOTICE for ivy.
          Hide
          Hoss Man added a comment -

          FWIW.... i did a completley clean checkout of the lucene3930 (r1306662) and got the following build failure trying to run "ant clean test" from the top level.

          the ivy bootstraping doesn't seem to play nicely with the license checker...

          hossman@bester:~/lucene/branch_lucene3930$ ant clean test
          Buildfile: build.xml
          
          clean:
          
          clean:
          
          clean:
          
          clean:
               [echo] Building analyzers-common...
          
          clean:
               [echo] Building analyzers-icu...
          
          clean:
               [echo] Building analyzers-kuromoji...
          
          clean:
               [echo] Building analyzers-morfologik...
          
          clean:
               [echo] Building analyzers-phonetic...
          
          clean:
               [echo] Building analyzers-smartcn...
          
          clean:
               [echo] Building analyzers-stempel...
          
          clean:
               [echo] Building analyzers-uima...
          
          clean:
               [echo] Building benchmark...
          
          clean:
               [echo] Building facet...
          
          clean:
               [echo] Building grouping...
          
          clean:
               [echo] Building join...
          
          clean:
               [echo] Building queries...
          
          clean:
               [echo] Building queryparser...
          
          clean:
               [echo] Building spatial...
          
          clean:
               [echo] Building suggest...
          
          clean:
               [echo] Building solr...
          
          clean:
          
          validate:
          
          compile-tools:
          
          download-ivy:
              [mkdir] Created dir: /home/hossman/lucene/branch_lucene3930/lucene/ivy
               [echo] 
               [echo]       NOTE: You do not have ivy installed, so downloading it...
               [echo]       If you make lots of checkouts, download ivy-2.2.0.jar yourself 
               [echo]       and set ivy.jar.file to this in your ~/build.properties
               [echo]       
                [get] Getting: http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar
                [get] To: /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar
          
          install-ivy:
          
          resolve:
          [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ ::
          [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
          
          init:
          
          compile-core:
              [mkdir] Created dir: /home/hossman/lucene/branch_lucene3930/lucene/build/tools/classes/java
              [javac] Compiling 2 source files to /home/hossman/lucene/branch_lucene3930/lucene/build/tools/classes/java
               [copy] Copying 1 file to /home/hossman/lucene/branch_lucene3930/lucene/build/tools/classes/java
          
          validate:
               [echo] License check under: /home/hossman/lucene/branch_lucene3930/lucene
           [licenses] MISSING LICENSE for the following file:
           [licenses]   /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar
           [licenses]   Expected locations below:
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-ASL.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-BSD.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-BSD_LIKE.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-CDDL.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-CPL.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-EPL.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-MIT.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-MPL.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-PD.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-SUN.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-COMPOUND.txt
           [licenses]   => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-FAKE.txt
           [licenses] Scanned 1 JAR file(s) for licenses (in 0.31s.), 1 error(s).
          
          BUILD FAILED
          /home/hossman/lucene/branch_lucene3930/build.xml:42: The following error occurred while executing this line:
          /home/hossman/lucene/branch_lucene3930/lucene/build.xml:178: The following error occurred while executing this line:
          /home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml:22: License check failed. Check the logs.
          
          Total time: 5 seconds
          
          Show
          Hoss Man added a comment - FWIW.... i did a completley clean checkout of the lucene3930 (r1306662) and got the following build failure trying to run "ant clean test" from the top level. the ivy bootstraping doesn't seem to play nicely with the license checker... hossman@bester:~/lucene/branch_lucene3930$ ant clean test Buildfile: build.xml clean: clean: clean: clean: [echo] Building analyzers-common... clean: [echo] Building analyzers-icu... clean: [echo] Building analyzers-kuromoji... clean: [echo] Building analyzers-morfologik... clean: [echo] Building analyzers-phonetic... clean: [echo] Building analyzers-smartcn... clean: [echo] Building analyzers-stempel... clean: [echo] Building analyzers-uima... clean: [echo] Building benchmark... clean: [echo] Building facet... clean: [echo] Building grouping... clean: [echo] Building join... clean: [echo] Building queries... clean: [echo] Building queryparser... clean: [echo] Building spatial... clean: [echo] Building suggest... clean: [echo] Building solr... clean: validate: compile-tools: download-ivy: [mkdir] Created dir: /home/hossman/lucene/branch_lucene3930/lucene/ivy [echo] [echo] NOTE: You do not have ivy installed, so downloading it... [echo] If you make lots of checkouts, download ivy-2.2.0.jar yourself [echo] and set ivy.jar.file to this in your ~/build.properties [echo] [get] Getting: http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar [get] To: /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar install-ivy: resolve: [ivy:retrieve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [ivy:retrieve] :: loading settings :: url = jar:file:/home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar!/org/apache/ivy/core/settings/ivysettings.xml init: compile-core: [mkdir] Created dir: /home/hossman/lucene/branch_lucene3930/lucene/build/tools/classes/java [javac] Compiling 2 source files to /home/hossman/lucene/branch_lucene3930/lucene/build/tools/classes/java [copy] Copying 1 file to /home/hossman/lucene/branch_lucene3930/lucene/build/tools/classes/java validate: [echo] License check under: /home/hossman/lucene/branch_lucene3930/lucene [licenses] MISSING LICENSE for the following file: [licenses] /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-2.2.0.jar [licenses] Expected locations below: [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-ASL.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-BSD.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-BSD_LIKE.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-CDDL.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-CPL.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-EPL.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-MIT.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-MPL.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-PD.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-SUN.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-COMPOUND.txt [licenses] => /home/hossman/lucene/branch_lucene3930/lucene/ivy/ivy-LICENSE-FAKE.txt [licenses] Scanned 1 JAR file(s) for licenses (in 0.31s.), 1 error(s). BUILD FAILED /home/hossman/lucene/branch_lucene3930/build.xml:42: The following error occurred while executing this line: /home/hossman/lucene/branch_lucene3930/lucene/build.xml:178: The following error occurred while executing this line: /home/hossman/lucene/branch_lucene3930/lucene/tools/custom-tasks.xml:22: License check failed. Check the logs. Total time: 5 seconds
          Hide
          Robert Muir added a comment -

          Now we have the problematic patched jars and the renamed unreleased jars.

          I will start with the patched ones: these are the worst case.
          I will look to see if we can have our patch file sitting in svn,
          we download the sources (not jar), patch with the patch file,
          and compile the thing up.

          Show
          Robert Muir added a comment - Now we have the problematic patched jars and the renamed unreleased jars. I will start with the patched ones: these are the worst case. I will look to see if we can have our patch file sitting in svn, we download the sources (not jar), patch with the patch file, and compile the thing up.
          Hide
          Robert Muir added a comment -

          Thanks for all the help!!!!!!!

          Show
          Robert Muir added a comment - Thanks for all the help!!!!!!!
          Hide
          Chris Male added a comment -

          Patch without pattern override. I'll pick up where I left off in the AM.

          Show
          Chris Male added a comment - Patch without pattern override. I'll pick up where I left off in the AM.
          Hide
          Robert Muir added a comment -

          I think the patch is fine, but it doesn't need to override ivy.retrieve.pattern,
          example/lib is already its default.

          Show
          Robert Muir added a comment - I think the patch is fine, but it doesn't need to override ivy.retrieve.pattern, example/lib is already its default.
          Hide
          Chris Male added a comment -

          Lets defer any unnecessary refactoring (even if it has benefits) to other issues.

          Absolutely, just adding more information to chew over here.

          Show
          Chris Male added a comment - Lets defer any unnecessary refactoring (even if it has benefits) to other issues. Absolutely, just adding more information to chew over here.
          Hide
          Robert Muir added a comment -

          Lets defer any unnecessary refactoring (even if it has benefits) to other issues.

          I think we want the minimal (reasonable) patch here that works as close as possible
          to the ant build: given the incubator discussion about jars, unfortunately I think
          we need to backport this to 3.6.

          So its important to keep everything as close as possible to what it was before so that
          packaging tasks, etc work.

          Seeing how things got working for solr I think we may want to revert even the test-framework/lib
          and go back to lucene/lib intentionally just for this purpose: I'll look into this.

          Show
          Robert Muir added a comment - Lets defer any unnecessary refactoring (even if it has benefits) to other issues. I think we want the minimal (reasonable) patch here that works as close as possible to the ant build: given the incubator discussion about jars, unfortunately I think we need to backport this to 3.6. So its important to keep everything as close as possible to what it was before so that packaging tasks, etc work. Seeing how things got working for solr I think we may want to revert even the test-framework/lib and go back to lucene/lib intentionally just for this purpose: I'll look into this.
          Hide
          Chris Male added a comment -

          This has a few other benefits. We can move the other targets related to the example to example/build.xml and tie in the resolve target.

          Show
          Chris Male added a comment - This has a few other benefits. We can move the other targets related to the example to example/build.xml and tie in the resolve target.
          Hide
          Chris Male added a comment -

          Patch for addressing Solr's example/lib jars.
          Adds a build.xml and ivy.xml to solr/example

          I haven't committed this since I'm unsure its the best way forward. I've explored other options with ivy (through the use of confs for example) and there doesn't seem to be any other clean way.

          Show
          Chris Male added a comment - Patch for addressing Solr's example/lib jars. Adds a build.xml and ivy.xml to solr/example I haven't committed this since I'm unsure its the best way forward. I've explored other options with ivy (through the use of confs for example) and there doesn't seem to be any other clean way.
          Hide
          Robert Muir added a comment -

          they should be moved. also test-framework/lib should get an svn:ignore of *.jar

          Show
          Robert Muir added a comment - they should be moved. also test-framework/lib should get an svn:ignore of *.jar
          Hide
          Chris Male added a comment -

          Should I move the notice files from lucene/lib to lucene/test-framework/lib?

          Show
          Chris Male added a comment - Should I move the notice files from lucene/lib to lucene/test-framework/lib?
          Hide
          Robert Muir added a comment -

          Committed Uwe's patch for dist-maven. I also added OOM Protection (©), but its still broken
          for the modules/analysis which recurses into all the submodules. This needs to be fixed to
          work like contrib-crawl/modules top-level crawl. But this is nothing new and really unrelated
          to this issue...

          Show
          Robert Muir added a comment - Committed Uwe's patch for dist-maven. I also added OOM Protection (©), but its still broken for the modules/analysis which recurses into all the submodules. This needs to be fixed to work like contrib-crawl/modules top-level crawl. But this is nothing new and really unrelated to this issue...
          Hide
          Chris Male added a comment -

          Great stuff Robert. I've come into the discussion a little late but I'll help where I can.

          bogusly every contrib/module currently needs an ivy.xml (even with no dependencies). I'd like to remove this but i just havent mucked with it yet.

          I don't think this is so bad? It shows that the contrib/module is properly configured and that the ivy integration was 'done' when the module was created. I think it might also help IDEs if every module is consistent.

          Show
          Chris Male added a comment - Great stuff Robert. I've come into the discussion a little late but I'll help where I can. bogusly every contrib/module currently needs an ivy.xml (even with no dependencies). I'd like to remove this but i just havent mucked with it yet. I don't think this is so bad? It shows that the contrib/module is properly configured and that the ivy integration was 'done' when the module was created. I think it might also help IDEs if every module is consistent.
          Show
          Robert Muir added a comment - Created branch: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene3930
          Hide
          Robert Muir added a comment -

          I will make a heavy committing branch for this issue. I think we can solve this soon!

          Show
          Robert Muir added a comment - I will make a heavy committing branch for this issue. I think we can solve this soon!
          Hide
          Robert Muir added a comment -

          updated patch:

          • core+contrib+modules work. though we need to remove jars and move to dependencies.
          • i found the source of the maven-ant-tasks OOM that hits us in prepare-release. It hit me here too. The problem is that you should not allow the taskdef to run in dist-maven if it already ran. so the one for ivy has unless=ivy.uptodate (which sits in that uptodateAndCompiled propset). Finally i ensured the modules 'crawl' passes this around. We should fix dist-maven the same way.
          • bogusly every contrib/module currently needs an ivy.xml (even with no dependencies). I'd like to remove this but i just havent mucked with it yet.
          Show
          Robert Muir added a comment - updated patch: core+contrib+modules work. though we need to remove jars and move to dependencies. i found the source of the maven-ant-tasks OOM that hits us in prepare-release. It hit me here too. The problem is that you should not allow the taskdef to run in dist-maven if it already ran. so the one for ivy has unless=ivy.uptodate (which sits in that uptodateAndCompiled propset). Finally i ensured the modules 'crawl' passes this around. We should fix dist-maven the same way. bogusly every contrib/module currently needs an ivy.xml (even with no dependencies). I'd like to remove this but i just havent mucked with it yet.
          Hide
          Robert Muir added a comment -

          Thanks: i will look at maven-ant-tasks. I have dependent tasks working (e.g. all of lucene+contrib+modules) currently... patch coming soon.

          Show
          Robert Muir added a comment - Thanks: i will look at maven-ant-tasks. I have dependent tasks working (e.g. all of lucene+contrib+modules) currently... patch coming soon.
          Hide
          Uwe Schindler added a comment -

          Here the patch with the maven-ant-tasks. It works theoretcally, but the dependent tasks dont completely work at the moment.

          Show
          Uwe Schindler added a comment - Here the patch with the maven-ant-tasks. It works theoretcally, but the dependent tasks dont completely work at the moment.
          Hide
          Uwe Schindler added a comment -

          I already implemented it

          Show
          Uwe Schindler added a comment - I already implemented it
          Hide
          Robert Muir added a comment -

          so you could simply fetch it with ivy and make it available to ant using <ivy:cachepatch classpathref="foobar"/> and then load the targets from the classpath in ref="foobar".

          I'm not gonna do that now: thats an optimization. Various ant tasks expect libs to be in lib/, so I want to populate lib/, not rewrite the build system.

          Show
          Robert Muir added a comment - so you could simply fetch it with ivy and make it available to ant using <ivy:cachepatch classpathref="foobar"/> and then load the targets from the classpath in ref="foobar". I'm not gonna do that now: thats an optimization. Various ant tasks expect libs to be in lib/, so I want to populate lib/, not rewrite the build system.
          Hide
          Uwe Schindler added a comment - - edited

          so you could simply fetch it with ivy and make it available to ant using <ivy:cachepatch pathid="foobar"/> and then load the targets from the classpath in classpathref="foobar".

          Like that:

          <ivy:cachepath organisation="emma" module="emma" revision="2.0.4217" inline="true" conf="ant" pathid="emma.classpath"/>
          <taskdef resource="emma_ant.properties" classpathref="emma.classpath" /> 
          

          Thanks!

          Show
          Uwe Schindler added a comment - - edited so you could simply fetch it with ivy and make it available to ant using <ivy:cachepatch pathid="foobar"/> and then load the targets from the classpath in classpathref="foobar". Like that: <ivy:cachepath organisation= "emma" module= "emma" revision= "2.0.4217" inline= "true" conf= "ant" pathid= "emma.classpath" /> <taskdef resource= "emma_ant.properties" classpathref= "emma.classpath" /> Thanks!
          Hide
          Steve Rowe added a comment -

          I have no idea what uses this maven-ant-tasks.jar thats still there, or where to fetch it from, but maybe Steve knows

          maven-ant-tasks.jar is used by generate-maven-artifacts / dist-maven / m2-deploy* to deploy Maven artifacts; these are used by the Jenkins Maven builds to deploy snapshots to the Apache Snapshot repository, and up through the most recent release, to create Maven release artifacts to be published on Maven central.

          I unpacked the .jar and found a POM under META-INF, and it says its groupId:artifactId:version coordinates are org.apache.maven:maven-ant-tasks:2.1.3, which is available from the Maven Central repository at http://repo1.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.3/.

          Show
          Steve Rowe added a comment - I have no idea what uses this maven-ant-tasks.jar thats still there, or where to fetch it from, but maybe Steve knows maven-ant-tasks.jar is used by generate-maven-artifacts / dist-maven / m2-deploy* to deploy Maven artifacts; these are used by the Jenkins Maven builds to deploy snapshots to the Apache Snapshot repository, and up through the most recent release, to create Maven release artifacts to be published on Maven central. I unpacked the .jar and found a POM under META-INF, and it says its groupId:artifactId:version coordinates are org.apache.maven:maven-ant-tasks:2.1.3 , which is available from the Maven Central repository at http://repo1.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.3/ .
          Hide
          Robert Muir added a comment -

          prototype patch for the lucene core.

          You dont need to do anything special for this patch to work: 'ant test' etc works as usual (though only for lucene-core!!!)

          i nuked all the libs in lucene/lib/ (junit, ant, ant-junit).

          These are instead dependencies of the test-framework.

          I have no idea what uses this maven-ant-tasks.jar thats still there, or where to fetch it from, but maybe Steve knows

          Show
          Robert Muir added a comment - prototype patch for the lucene core. You dont need to do anything special for this patch to work: 'ant test' etc works as usual (though only for lucene-core!!!) i nuked all the libs in lucene/lib/ (junit, ant, ant-junit). These are instead dependencies of the test-framework. I have no idea what uses this maven-ant-tasks.jar thats still there, or where to fetch it from, but maybe Steve knows

            People

            • Assignee:
              Robert Muir
              Reporter:
              Robert Muir
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development