Lucene - Core
  1. Lucene - Core
  2. LUCENE-885

clean up build files so contrib tests are run more easily

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.2
    • Component/s: general/build
    • Labels:
      None
    • Lucene Fields:
      Patch Available

      Description

      Per mailing list discussion...

      http://www.nabble.com/Tests%2C-Contribs%2C-and-Releases-tf3768924.html#a10655448

      Tests for contribs should be run when "ant test" is used, existing "test" target renamed to "test-core"

      1. LUCENE-885.patch
        13 kB
        Hoss Man
      2. LUCENE-885.patch
        6 kB
        Hoss Man
      3. LUCENE-885-pt2.patch
        2 kB
        Steven Parkes

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          Whileworking on this, i fixed a few general issues with the way some contribs could be built/tested easily from their own directory, but not as part of hte larger "build-contrib" "test-contrib" targets, items included in this patch...

          1) change the main "test" target to "test-core" and add a new "test" target which depends on test and test-contrib
          2) change "test.classpath" reference to include core test classes (which some contrib test classes have compile dependencies on)
          3) make xml-query-parser's dependency on the queries contrib more standardized so the queries jar is forcably built no matter how xml-query-parser is built
          4) fix MemoryIndexTest so it can find the input files it's looking at regardless of what directory the test is run from.
          5) make the generate-test-reports target work both for individual contribs as well as for hte project as a whole
          6) make contrib-crawl fail by default if the target fails for any of the contribs (so build-contrib now fails immediately if a contrib fails to build)
          7) make test-contrib delay any failure until all contribs have been tested – this makes it possible to run "ant generate-test-reports" after a failed "ant test" and see all of the failures.

          Note that i only modified contribs in cases where they had problems related to their expectations about being built or tested in a specific directory ... as it stands there are still some test errors/failures...

          #a) TestPrecedenceQueryParser has 3 failures
          #b) spellchecker has one test failure (i believe there is already a patch in another issue)
          #c) contrib/db/bdb has dependencies on a native library

          #a and #b are related to functionality ... i have no qualms about them being broken and preventing builds until decisisons are made about how to fix them correctly.

          #c on the other hand raises some interesting questions: should we skip those tests unless the native library is present? ... i'm not even sure how to do that (<available> on a classpath resource i understand, java.library.path i don't). Are committers preparing official releases expected to have those native libraries in order to sign off on the build?

          I'm happy to commit this as is and let the nightly builds be broken until we deal with this ... but it would be nice if at least one other person could try it out an verify that it works for them as well.

          Show
          Hoss Man added a comment - Whileworking on this, i fixed a few general issues with the way some contribs could be built/tested easily from their own directory, but not as part of hte larger "build-contrib" "test-contrib" targets, items included in this patch... 1) change the main "test" target to "test-core" and add a new "test" target which depends on test and test-contrib 2) change "test.classpath" reference to include core test classes (which some contrib test classes have compile dependencies on) 3) make xml-query-parser's dependency on the queries contrib more standardized so the queries jar is forcably built no matter how xml-query-parser is built 4) fix MemoryIndexTest so it can find the input files it's looking at regardless of what directory the test is run from. 5) make the generate-test-reports target work both for individual contribs as well as for hte project as a whole 6) make contrib-crawl fail by default if the target fails for any of the contribs (so build-contrib now fails immediately if a contrib fails to build) 7) make test-contrib delay any failure until all contribs have been tested – this makes it possible to run "ant generate-test-reports" after a failed "ant test" and see all of the failures. Note that i only modified contribs in cases where they had problems related to their expectations about being built or tested in a specific directory ... as it stands there are still some test errors/failures... #a) TestPrecedenceQueryParser has 3 failures #b) spellchecker has one test failure (i believe there is already a patch in another issue) #c) contrib/db/bdb has dependencies on a native library #a and #b are related to functionality ... i have no qualms about them being broken and preventing builds until decisisons are made about how to fix them correctly. #c on the other hand raises some interesting questions: should we skip those tests unless the native library is present? ... i'm not even sure how to do that (<available> on a classpath resource i understand, java.library.path i don't). Are committers preparing official releases expected to have those native libraries in order to sign off on the build? I'm happy to commit this as is and let the nightly builds be broken until we deal with this ... but it would be nice if at least one other person could try it out an verify that it works for them as well.
          Hide
          Paul Elschot added a comment -

          The patch applies cleanly on my trunk working copy (using patch -p0 < *885*patch, unix).

          Sure enough ant test-contrib fails in the trunk.
          (The memory index test was taking quite a bit of time.)

          Also, in contrib/surround ant test fails, so I'll have further look at how to better combine contrib/surround/build.xml
          and the trunk build.xml and related files.

          What is the preferred way to run the unit tests for a single contrib?
          From the trunk build.xml or from the contrib/*/build.xml ?

          Show
          Paul Elschot added a comment - The patch applies cleanly on my trunk working copy (using patch -p0 < *885*patch, unix). Sure enough ant test-contrib fails in the trunk. (The memory index test was taking quite a bit of time.) Also, in contrib/surround ant test fails, so I'll have further look at how to better combine contrib/surround/build.xml and the trunk build.xml and related files. What is the preferred way to run the unit tests for a single contrib? From the trunk build.xml or from the contrib/*/build.xml ?
          Hide
          Hoss Man added a comment -

          Paul: i can't reproduce the problem you describe..

          > Also, in contrib/surround ant test fails, ...

          using this patch, i see all (65) tests (in 3 test classes) in the org.apache.lucene.queryParser.surround.query package pass regardless of whether I run "ant test" from the top level, or from contrib/surround ... can you attach a log of exactly hat you're doing when you get failures?

          I'm also not sure i understand your question...

          > What is the preferred way to run the unit tests for a single contrib?
          > From the trunk build.xml or from the contrib/*/build.xml ?

          ....from the main build.xml, "ant test-contrib" will run the tests for all constribs ... if you only want to run test for an individual contrib, the only way i know to do that is from the contrib's directory ... your question of "preferred" way suggests there is another way to run tests for a single contrib, and i'm not sure what you're referring to.

          One final note: testing this patch on a differnet computer today, i'm getting some additional contrib test errors in org.apache.lucene.gdata.server.registry and org.apache.lucene.gdata.servlet.handler (i thought i was using 1.5 yesterday but perhaps not). still need to investigate if these are legitimate code failures, or problems triggered by how the tests are run.

          Show
          Hoss Man added a comment - Paul: i can't reproduce the problem you describe.. > Also, in contrib/surround ant test fails, ... using this patch, i see all (65) tests (in 3 test classes) in the org.apache.lucene.queryParser.surround.query package pass regardless of whether I run "ant test" from the top level, or from contrib/surround ... can you attach a log of exactly hat you're doing when you get failures? I'm also not sure i understand your question... > What is the preferred way to run the unit tests for a single contrib? > From the trunk build.xml or from the contrib/*/build.xml ? ....from the main build.xml, "ant test-contrib" will run the tests for all constribs ... if you only want to run test for an individual contrib, the only way i know to do that is from the contrib's directory ... your question of "preferred" way suggests there is another way to run tests for a single contrib, and i'm not sure what you're referring to. One final note: testing this patch on a differnet computer today, i'm getting some additional contrib test errors in org.apache.lucene.gdata.server.registry and org.apache.lucene.gdata.servlet.handler (i thought i was using 1.5 yesterday but perhaps not). still need to investigate if these are legitimate code failures, or problems triggered by how the tests are run.
          Hide
          Paul Elschot added a comment -

          > Paul: i can't reproduce the problem you describe..

          That's good news, especially with you having more tests pass.
          I may have to restart from a fresh checkout instead of using my working copy.

          As for the preferred way of running single contrib tests, a better way to phrase my question would probably be: What facilities of the lucene build.xml can be used (imported) in the build.xml of a single contrib?

          I remember creating the build.xml file for surround by starting from the lucene build.xml.
          That means there is probably some redundancy left in the current build.xml of surround with respect to the lucene build.xml, and I'd like to remove that redundancy.

          Show
          Paul Elschot added a comment - > Paul: i can't reproduce the problem you describe.. That's good news, especially with you having more tests pass. I may have to restart from a fresh checkout instead of using my working copy. As for the preferred way of running single contrib tests, a better way to phrase my question would probably be: What facilities of the lucene build.xml can be used (imported) in the build.xml of a single contrib? I remember creating the build.xml file for surround by starting from the lucene build.xml. That means there is probably some redundancy left in the current build.xml of surround with respect to the lucene build.xml, and I'd like to remove that redundancy.
          Hide
          Paul Elschot added a comment -

          Ok. Two mistakes by me.
          Turns out I had made an addition that would not compile, and then the tests won't work either.

          Also the build.xml file of surround is just about as empty as possible, so that is no problem
          either.

          Sorry for the noise.

          Show
          Paul Elschot added a comment - Ok. Two mistakes by me. Turns out I had made an addition that would not compile, and then the tests won't work either. Also the build.xml file of surround is just about as empty as possible, so that is no problem either. Sorry for the noise.
          Hide
          Hoss Man added a comment -

          revised patch addresses two issues...

          1) the gadta tests had similar issues to contrib/memory – assumptions about CWD for purposes of opening files. i generalized the solution i put in for contrib/memory to create a new "lucene.common.dir" system property when making the <junit> call so that tests would always have a root directory to reference when opening files.

          2) i added a new SanityLoadLibrary class to the contrib/db/bdb test directory ... it's not a unity test, but a minimal java app for generateing an error if the native library needed for the package can't be loaded — if it doesn't exit cleanly, tests for that package will be skipped.

          NOTE: TestPrecedenceQueryParser still has test failures, which occur regardless of how the test is run — i'm making no attempt to fix those since they are not related to the build system.

          I'm comfortable committing this as is in a few days, but feedback is always appreciated. in particular, it would be nice if someone with the BDB native library installed could sanity test that the build system correctly runs the test in the presence of that library.

          Show
          Hoss Man added a comment - revised patch addresses two issues... 1) the gadta tests had similar issues to contrib/memory – assumptions about CWD for purposes of opening files. i generalized the solution i put in for contrib/memory to create a new "lucene.common.dir" system property when making the <junit> call so that tests would always have a root directory to reference when opening files. 2) i added a new SanityLoadLibrary class to the contrib/db/bdb test directory ... it's not a unity test, but a minimal java app for generateing an error if the native library needed for the package can't be loaded — if it doesn't exit cleanly, tests for that package will be skipped. NOTE: TestPrecedenceQueryParser still has test failures, which occur regardless of how the test is run — i'm making no attempt to fix those since they are not related to the build system. I'm comfortable committing this as is in a few days, but feedback is always appreciated. in particular, it would be nice if someone with the BDB native library installed could sanity test that the build system correctly runs the test in the presence of that library.
          Hide
          Michael Busch added a comment -

          Hoss,

          This looks good! On my system (Win XP) all contribs (except db, since
          I don't have BDB installed) build fine and the only failing tests are
          in TestPrecedenceQueryParser as you mentioned already.
          spell and surround are fine, too.

          A comment about the PrecendenceQueryParser (PQP): I took a look into the
          commit history and found the following two comments:

          "Minor tweaks to the new proposed query parser. There are still some
          failing tests, as some were added to show expectations not yet met. The
          main missing piece is getting NOT precedence accounted for."

          and

          "move PrecedenceQueryParser to contrib/misc until the kinks are
          worked out"

          Both comments are from early 2005. Since then nobody has changed the
          code, so I think we should exclude PQP from the build temporarily
          until someone starts working on it again?

          Show
          Michael Busch added a comment - Hoss, This looks good! On my system (Win XP) all contribs (except db, since I don't have BDB installed) build fine and the only failing tests are in TestPrecedenceQueryParser as you mentioned already. spell and surround are fine, too. A comment about the PrecendenceQueryParser (PQP): I took a look into the commit history and found the following two comments: "Minor tweaks to the new proposed query parser. There are still some failing tests, as some were added to show expectations not yet met. The main missing piece is getting NOT precedence accounted for." and "move PrecedenceQueryParser to contrib/misc until the kinks are worked out" Both comments are from early 2005. Since then nobody has changed the code, so I think we should exclude PQP from the build temporarily until someone starts working on it again?
          Hide
          Erik Hatcher added a comment -

          PQP was a hack I made long ago to mainly show how QP could be possibly improved. I'm fine with that class being removed altogether, or the failing tests commented out. I don't use that class personally.

          Show
          Erik Hatcher added a comment - PQP was a hack I made long ago to mainly show how QP could be possibly improved. I'm fine with that class being removed altogether, or the failing tests commented out. I don't use that class personally.
          Hide
          Hoss Man added a comment -

          Committed revision 542769.

          ...this included two minor tweaks to the bdb stuff from what was in the last patch...
          1) fixed property name from no-dbd-lib to no-bdb-lib
          2) modified both the bdb and bdb-je tests to use the temp directory specified as a system prop when the <junit> task is run to keep them from polluting the root directory.

          Show
          Hoss Man added a comment - Committed revision 542769. ...this included two minor tweaks to the bdb stuff from what was in the last patch... 1) fixed property name from no-dbd-lib to no-bdb-lib 2) modified both the bdb and bdb-je tests to use the temp directory specified as a system prop when the <junit> task is run to keep them from polluting the root directory.
          Hide
          Hoss Man added a comment -

          Officially reopening this bug as i have discovered that it causes the build to fail on java 1.4

          the problem is that the contrib-crawl logic used by build-contrib and test-contrib is ignorant of the "skip 1.5 contribs" logic used in the javadocs (it is a javadoc specific property) and the individiaul 1.5 contribs (ie: gdata) assume that ify ou are trying to build them, you must have 1.5.

          patch is already ready to make the property more global, and to make the targets in the gdata build.xml act as NOOPs (echoing a message) based on the value ... just doing some more testing now before committing.

          Show
          Hoss Man added a comment - Officially reopening this bug as i have discovered that it causes the build to fail on java 1.4 the problem is that the contrib-crawl logic used by build-contrib and test-contrib is ignorant of the "skip 1.5 contribs" logic used in the javadocs (it is a javadoc specific property) and the individiaul 1.5 contribs (ie: gdata) assume that ify ou are trying to build them, you must have 1.5. patch is already ready to make the property more global, and to make the targets in the gdata build.xml act as NOOPs (echoing a message) based on the value ... just doing some more testing now before committing.
          Hide
          Hoss Man added a comment -

          Committed revision 543257.

          Compilation and test of the entire tree should work fine now under 1.4 ... note that gdata doesn't actually run it's tests (even under 1.5) because of LUCENE-899 ... but this problem predates any work done for this issue, so i'm not going to look into it at this time as it relates to bugs in a specific contrib, and not in changes made to facilitate the building/testing of contribs.

          Show
          Hoss Man added a comment - Committed revision 543257. Compilation and test of the entire tree should work fine now under 1.4 ... note that gdata doesn't actually run it's tests (even under 1.5) because of LUCENE-899 ... but this problem predates any work done for this issue, so i'm not going to look into it at this time as it relates to bugs in a specific contrib, and not in changes made to facilitate the building/testing of contribs.
          Hide
          Steven Parkes added a comment -

          Looking at the current build (r545324) it looks like the some contrib failures are getting swallowed. Things like lucli are throwing errors along the lines of

          [subant] /home/barronpark/smparkes/work/lucene/trunk/common-build.xml:366: srcdir "/home/barronpark/smparkes/work/lucene/trunk/contrib/lucli/src/test" does not exist!

          but these don't make it back up to the top level status.

          It looks like the current state will bubble up junit failures, but maybe not build failures? There are some flag files being created, but they aren't checking this step, I don't think.

          Show
          Steven Parkes added a comment - Looking at the current build (r545324) it looks like the some contrib failures are getting swallowed. Things like lucli are throwing errors along the lines of [subant] /home/barronpark/smparkes/work/lucene/trunk/common-build.xml:366: srcdir "/home/barronpark/smparkes/work/lucene/trunk/contrib/lucli/src/test" does not exist! but these don't make it back up to the top level status. It looks like the current state will bubble up junit failures, but maybe not build failures? There are some flag files being created, but they aren't checking this step, I don't think.
          Hide
          Hoss Man added a comment -

          the flag files are only used when running the tests (i created them so that reports could be generated on all tests even if one failure occured in one contrib). By default the contrib-crawl target will fail if any of the subant calls fail - test-contrib overrides this (hence it needs the flag files) but build-contrib does not so i'm not sure why you would get an error like that. any failure from any contrib in should fail build-contrib.

          Show
          Hoss Man added a comment - the flag files are only used when running the tests (i created them so that reports could be generated on all tests even if one failure occured in one contrib). By default the contrib-crawl target will fail if any of the subant calls fail - test-contrib overrides this (hence it needs the flag files) but build-contrib does not so i'm not sure why you would get an error like that. any failure from any contrib in should fail build-contrib.
          Hide
          Steven Parkes added a comment -

          Ah, I see.

          Should, then, test-contrib depend on build-contrib, rather than compile-test?

          That will flush out these issues (and, as a side effect, immediately break the build ...)

          Show
          Steven Parkes added a comment - Ah, I see. Should, then, test-contrib depend on build-contrib, rather than compile-test? That will flush out these issues (and, as a side effect, immediately break the build ...)
          Hide
          Steven Parkes added a comment -

          Whoops, the primary failure was because I wasn't up to date.

          Actually, making test-contrib rely on build-contrib doesn't fix it either. It's "test-compile-contrib" (if you will) that fails and rather being contrib-crawled, that's only done as the target of "test" in each contrib directory, at which point, it's running in the protected contrib-crawl.

          Easy enough to lift this loop into another target, e.g., build-contrib-test. And that will start surfacing errors, which I can work through.

          I've got patches. Want them on this JIRA or another?

          Show
          Steven Parkes added a comment - Whoops, the primary failure was because I wasn't up to date. Actually, making test-contrib rely on build-contrib doesn't fix it either. It's "test-compile-contrib" (if you will) that fails and rather being contrib-crawled, that's only done as the target of "test" in each contrib directory, at which point, it's running in the protected contrib-crawl. Easy enough to lift this loop into another target, e.g., build-contrib-test. And that will start surfacing errors, which I can work through. I've got patches. Want them on this JIRA or another?
          Hide
          Steven Parkes added a comment -

          This patch file removes the swallowed failures by doing more of the build steps out of the guarded loop (that only detects junit failures, not others).

          In addition the patch, need to create and "svn add"
          A contrib/wordnet/src/test
          A contrib/similarity/src/test
          A contrib/lucli/src/test

          This makes it possible to actually run the gdata-server tests, which were not running before. I have seen one case where it hung, but maybe that was a fluke.

          Show
          Steven Parkes added a comment - This patch file removes the swallowed failures by doing more of the build steps out of the guarded loop (that only detects junit failures, not others). In addition the patch, need to create and "svn add" A contrib/wordnet/src/test A contrib/similarity/src/test A contrib/lucli/src/test This makes it possible to actually run the gdata-server tests, which were not running before. I have seen one case where it hung, but maybe that was a fluke.
          Hide
          Steven Parkes added a comment -

          Renaming the patch to make it clear it adds to the previous, rather than replaces.

          Show
          Steven Parkes added a comment - Renaming the patch to make it clear it adds to the previous, rather than replaces.
          Hide
          Steven Parkes added a comment -

          Reopening to get these tweaks in.

          Show
          Steven Parkes added a comment - Reopening to get these tweaks in.
          Hide
          Hoss Man added a comment -

          Steven:

          Since changes with this issue number have already been committed and recorded in the CHANGES.txt file prior to the 2.2 branch, and since i'm not certain these additional changes will make it into the 2.2 branch (not my call to make) i think it would be better to open a new (linked) issue with a unique number for tracking purposes.

          I'll take care of that now.

          Show
          Hoss Man added a comment - Steven: Since changes with this issue number have already been committed and recorded in the CHANGES.txt file prior to the 2.2 branch, and since i'm not certain these additional changes will make it into the 2.2 branch (not my call to make) i think it would be better to open a new (linked) issue with a unique number for tracking purposes. I'll take care of that now.

            People

            • Assignee:
              Hoss Man
              Reporter:
              Hoss Man
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development