Solr
  1. Solr
  2. SOLR-3

create test harness and port TestApp to junit

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      To both encourage good internal development, and to make it easy for plugin developers to write unit tests of their own code I think we need a harness that makes it easy to unit test updates and queries against Solr (without needing a servlet container)

      Once we have this, i think we can/should also retire "TestApp" in favor of some JUnit tests (which would probably make more sense for other developers)

      Iv'e already started on this, i thought i'd have something to commit tonight, but i got distracted ... filing this bug as a tracker for now.

      1. TestBasicFunctionality.java
        5 kB
        Hoss Man
      2. TestHarness.java
        9 kB
        Hoss Man
      3. SOLR-3.zip
        18 kB
        Hoss Man
      There are no Sub-Tasks for this issue.

        Activity

        Hoss Man created issue -
        Hide
        Hoss Man added a comment -

        Here's what i've got so far, still a work in progress – but you can see the basic idea.

        I was trying to avoid putting anything junit specific in the TestHarness, but it definitely looks like having an abstract subclass of TestCase that provides some assert methods that can take in adds or requests/xpaths and validate directly would make hte tests easier to read.

        Any comments would be appreacited.

        Show
        Hoss Man added a comment - Here's what i've got so far, still a work in progress – but you can see the basic idea. I was trying to avoid putting anything junit specific in the TestHarness, but it definitely looks like having an abstract subclass of TestCase that provides some assert methods that can take in adds or requests/xpaths and validate directly would make hte tests easier to read. Any comments would be appreacited.
        Hoss Man made changes -
        Field Original Value New Value
        Attachment TestHarness.java [ 12324084 ]
        Attachment TestBasicFunctionality.java [ 12324085 ]
        Hide
        Yonik Seeley added a comment -

        Looking good Hoss!

        There are probably some opportunities to use Java5 stuff like varargs...
        doc("id",42,"subject","easy")

        Also what we really need is a way to dynamically add/create a schema, so one could add a new analysis filter,
        then define a fieldtype & field that uses it and test it out, without modifying some global testing schema.

        Show
        Yonik Seeley added a comment - Looking good Hoss! There are probably some opportunities to use Java5 stuff like varargs... doc("id",42,"subject","easy") Also what we really need is a way to dynamically add/create a schema, so one could add a new analysis filter, then define a fieldtype & field that uses it and test it out, without modifying some global testing schema.
        Hide
        Hoss Man added a comment -

        >There are probably some opportunities to use Java5 stuff like varargs...
        > doc("id",42,"subject","easy")

        ...i'm doing that in validateAddDoc (which calls makeSimpleDoc) ... are there other methods you think it also makes sense for?

        > Also what we really need is a way to dynamically add/create a schema, so one could add
        > a new analysis filter,
        > then define a fieldtype & field that uses it and test it out, without modifying some global
        > testing schema.

        i was operating under the assumption that for stuff like that, the best thing to do would be to have multiple sets of solrconfig/schema files ... for Solr itself there would probably be one big generic file that most tests could use, but any test that wanted to try something exotic would check in a new one (in a directory with the same name as the test class) and speciy it when constructing hte harness.

        the only flaw in that plan right now is that SolrCore only lets you specify a schema file path when you construct it, not a solrconfig, but i was going to tackle refactoring that once i had the basics up and running.

        Show
        Hoss Man added a comment - >There are probably some opportunities to use Java5 stuff like varargs... > doc("id",42,"subject","easy") ...i'm doing that in validateAddDoc (which calls makeSimpleDoc) ... are there other methods you think it also makes sense for? > Also what we really need is a way to dynamically add/create a schema, so one could add > a new analysis filter, > then define a fieldtype & field that uses it and test it out, without modifying some global > testing schema. i was operating under the assumption that for stuff like that, the best thing to do would be to have multiple sets of solrconfig/schema files ... for Solr itself there would probably be one big generic file that most tests could use, but any test that wanted to try something exotic would check in a new one (in a directory with the same name as the test class) and speciy it when constructing hte harness. the only flaw in that plan right now is that SolrCore only lets you specify a schema file path when you construct it, not a solrconfig, but i was going to tackle refactoring that once i had the basics up and running.
        Hide
        Hoss Man added a comment -

        zip file contains a bunch of code and a patch file that modifies the build.xml and XML.java

        the new code consists of:

        • the TestHarness from before (with a few additions) which should be usefull for interacting with a SolrCore independent of any specific testing framework
        • a new AbstractSolrTestCase that is designed to make it easy to write JUnit tests in particular.
        • a Sample test, and a short test of the basic Solr functions that demonstrate "best practices" of the AbstractSolrTestCase
        • a ConvertedLegacyTest which is a progromatic translation of src/apps/SolrTest/newtest.txt (the converstion perl script is included in the zip for refrence, but i wasn't planning on commiting it.)

        this represents 95% of what i think we need as far as a good testing framework moving forward: the last 5% being modifications to SolrConfig so that individual tests can use different solrconfig.xml files. Even without that, this code can be commited as is – tests will just have to share a solrconfig.xml for the time being, but they can use unique schema files.

        The only hitch with all of this, is that i seem to have a filehandle leak somewhere. Or at least i think i do ... running "ant legacyTest" works fine with an openfiles limit of 1024, but "ant junit" runs out of filehandles using the same limit (at 2048 the test runs fine). The index itself only contains about 100 files when this happens so i'm not sure what's going on.

        If anyone can help me spot the problem, i'd appreciate it.

        I'm going to take a look at it with some fresh eyes (and lsof) in the morning.

        Show
        Hoss Man added a comment - zip file contains a bunch of code and a patch file that modifies the build.xml and XML.java the new code consists of: the TestHarness from before (with a few additions) which should be usefull for interacting with a SolrCore independent of any specific testing framework a new AbstractSolrTestCase that is designed to make it easy to write JUnit tests in particular. a Sample test, and a short test of the basic Solr functions that demonstrate "best practices" of the AbstractSolrTestCase a ConvertedLegacyTest which is a progromatic translation of src/apps/SolrTest/newtest.txt (the converstion perl script is included in the zip for refrence, but i wasn't planning on commiting it.) this represents 95% of what i think we need as far as a good testing framework moving forward: the last 5% being modifications to SolrConfig so that individual tests can use different solrconfig.xml files. Even without that, this code can be commited as is – tests will just have to share a solrconfig.xml for the time being, but they can use unique schema files. The only hitch with all of this, is that i seem to have a filehandle leak somewhere. Or at least i think i do ... running "ant legacyTest" works fine with an openfiles limit of 1024, but "ant junit" runs out of filehandles using the same limit (at 2048 the test runs fine). The index itself only contains about 100 files when this happens so i'm not sure what's going on. If anyone can help me spot the problem, i'd appreciate it. I'm going to take a look at it with some fresh eyes (and lsof) in the morning.
        Hoss Man made changes -
        Attachment SOLR-3.zip [ 12325103 ]
        Hoss Man made changes -
        Assignee Hoss Man [ hossman ]
        Hide
        Hoss Man added a comment -

        filehandle leak was resolved by closing requests, code was commited.

        leaving issue open for now, pending sub-task to deal with solrconfig.xml loading, and a decission about wether we want to completely remove SolrTest or not. (either way, we should copy/move the config/schema used by the tests somewhere under the test directory, and change the working directory of the JUnit tests)

        Show
        Hoss Man added a comment - filehandle leak was resolved by closing requests, code was commited. leaving issue open for now, pending sub-task to deal with solrconfig.xml loading, and a decission about wether we want to completely remove SolrTest or not. (either way, we should copy/move the config/schema used by the tests somewhere under the test directory, and change the working directory of the JUnit tests)
        Hide
        Hoss Man added a comment -

        Everything I set out to do is done. Should this issue be resolved, or should SolrTest be removed from the repository?

        Does it serve any useful purpose now that we have the JUnit tests?

        Show
        Hoss Man added a comment - Everything I set out to do is done. Should this issue be resolved, or should SolrTest be removed from the repository? Does it serve any useful purpose now that we have the JUnit tests?
        Hide
        Yonik Seeley added a comment -

        Yes, I suppose SolrTest can be removed <nostalgic sniff...>

        Show
        Yonik Seeley added a comment - Yes, I suppose SolrTest can be removed <nostalgic sniff...>
        Hide
        Hoss Man added a comment -

        Well, the one thing SolrTest could do that we don't have a replacement for that SolrTest was usefull for is load testing SolrCore – it might make sense to leave that part in, but rip out the guts and replace it with using the SolrTestHarness.

        Show
        Hoss Man added a comment - Well, the one thing SolrTest could do that we don't have a replacement for that SolrTest was usefull for is load testing SolrCore – it might make sense to leave that part in, but rip out the guts and replace it with using the SolrTestHarness.
        Hide
        Hoss Man added a comment -

        In the interest of reducing possible confusion (and since i haven't been motivated to refactor SolrTest to use SolrTestHarness in the last 6 months) i think maybe I should just svn remove solr/src/apps and the build.xml references to it before we have an official release.

        we can always restore it in the future if there is a desire.

        objections?

        Show
        Hoss Man added a comment - In the interest of reducing possible confusion (and since i haven't been motivated to refactor SolrTest to use SolrTestHarness in the last 6 months) i think maybe I should just svn remove solr/src/apps and the build.xml references to it before we have an official release. we can always restore it in the future if there is a desire. objections?
        Hide
        Bertrand Delacretaz added a comment -

        No objection here, keeping the code clean is a Good Thing

        Show
        Bertrand Delacretaz added a comment - No objection here, keeping the code clean is a Good Thing
        Hide
        Yonik Seeley added a comment -

        > maybe I should just svn remove solr/src/apps

        +1

        Show
        Yonik Seeley added a comment - > maybe I should just svn remove solr/src/apps +1
        Hide
        Hoss Man added a comment -

        TestApp removed ... thus ends the longest "In Progress" Solr issue to date

        Show
        Hoss Man added a comment - TestApp removed ... thus ends the longest "In Progress" Solr issue to date
        Hoss Man made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hoss Man added a comment -

        This bug was modified as part of a bulk update using the criteria...

        • Marked ("Resolved" or "Closed") and "Fixed"
        • Had no "Fix Version" versions
        • Was listed in the CHANGES.txt for 1.1

        The Fix Version for all 38 issues found was set to 1.1, email notification
        was suppressed to prevent excessive email.

        For a list of all the issues modified, search jira comments for this
        (hopefully) unique string: 20080415hossman3

        Show
        Hoss Man added a comment - This bug was modified as part of a bulk update using the criteria... Marked ("Resolved" or "Closed") and "Fixed" Had no "Fix Version" versions Was listed in the CHANGES.txt for 1.1 The Fix Version for all 38 issues found was set to 1.1, email notification was suppressed to prevent excessive email. For a list of all the issues modified, search jira comments for this (hopefully) unique string: 20080415hossman3
        Hoss Man made changes -
        Fix Version/s 1.1.0 [ 12312234 ]
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Hoss Man
            Reporter:
            Hoss Man
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development