Solr
  1. Solr
  2. SOLR-2659

src/test-files/** should be moved under src/test-files/<module-name>/** for all Solr modules except core

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.4, 4.0-ALPHA
    • Fix Version/s: 3.4, 4.0-ALPHA
    • Component/s: Build
    • Labels:
      None

      Description

      SOLR-2452 split the solrj & common tests and test-files out from under solr/src/test{,-files} and placed them under solr/solrj/.

      Because IntelliJ's dependency scheme can't directly support the dependencies among the core/, solrj/, and test-framework/ internal modules, IntelliJ runs core/ and solrj/ tests under the monolithic IntelliJ "solr" module,

      As a result, when IntelliJ copies core/src/test-files/** and solrj/src/test-files/** to solr/build/solr-idea/classes/test/ (the test output directory), only one file from each same-named file pair can reside in the target directory, e.g. solr/conf/schema.xml. When same-named files differ between the two test-files/ directories, tests will fail. E.g.: LUCENE-2048 introduced a nopositions fieldType and a nopositionstext field into core/src/test-files/solr/conf/schema.xml, but not into the same-named file under solrj/src/test-files/, so when IntelliJ chooses the solrj version when copying resources, the core test that depends on the nopositionstext field (TestOmitPositions) will fail.

      I propose adding an extra directory level under solrj/src/test-files/: solrj/src/test-files/solrj/. That way, files from core/src/test-files/ can have the same names, but still co-exist when copied to the test output directory by IntelliJ.

      To maintain consistency, as well as avoid future naming conflicts, all other solr modules except core should switch to the same layout: src/test-files/<module-name>/*. Currently all contribs' solr homes are named src/test-files/solr-<module-name>/ - these directories should be renamed to src/test-files/<module-name>/solr.

      1. SOLR-2659.svn.movement.sh
        1 kB
        Steve Rowe
      2. SOLR-2659.patch
        9 kB
        Steve Rowe
      3. SOLR-2659.patch
        30 kB
        Steve Rowe
      4. SOLR-2659.part2.patch
        2 kB
        Steve Rowe

        Activity

        Hide
        Steve Rowe added a comment -

        Patch implementing the proposal.

        Before applying the patch, run this svn movement script on the cmdline (under a Bash shell):

        svn mkdir solr/solrj/src/test-files/solrj
        svn mv solr/solrj/src/test-files/{README,books.csv,docs1.xml,docs2.xml,sampleDateFacetResponse.xml,solr} solr/solrj/src/test-files/solrj
        
        Show
        Steve Rowe added a comment - Patch implementing the proposal. Before applying the patch, run this svn movement script on the cmdline (under a Bash shell): svn mkdir solr/solrj/src/test-files/solrj svn mv solr/solrj/src/test-files/{README,books.csv,docs1.xml,docs2.xml,sampleDateFacetResponse.xml,solr} solr/solrj/src/test-files/solrj
        Hide
        Robert Muir added a comment -

        maybe simpler/more consistent if its solr home is just solr-solrj? this is how the other ones work, e.g. solr-dih, solr-extraction, ...

        Show
        Robert Muir added a comment - maybe simpler/more consistent if its solr home is just solr-solrj? this is how the other ones work, e.g. solr-dih, solr-extraction, ...
        Hide
        Steve Rowe added a comment -

        maybe simpler/more consistent if its solr home is just solr-solrj? this is how the other ones work, e.g. solr-dih, solr-extraction, ...

        Unlike all of these, however, solrj/src/test-files/ has files that are at the top level, not under its solr home.

        Show
        Steve Rowe added a comment - maybe simpler/more consistent if its solr home is just solr-solrj? this is how the other ones work, e.g. solr-dih, solr-extraction, ... Unlike all of these, however, solrj/src/test-files/ has files that are at the top level, not under its solr home.
        Hide
        Steve Rowe added a comment -

        maybe simpler/more consistent if its solr home is just solr-solrj? this is how the other ones work, e.g. solr-dih, solr-extraction, ...

        Unlike all of these, however, solrj/src/test-files/ has files that are at the top level, not under its solr home.

        I was wrong: Solr contribs clustering, dih-extras, extraction, and uima all have files at the top level not under */src/test-files/solr-*/. These appear mostly to have different names, but in at least one case (solr-word.pdf), there is a same-named file in more than one module (extraction and dih-extras). So it would appear that the systemic problem is not fixed by just renaming the solr homes?

        Show
        Steve Rowe added a comment - maybe simpler/more consistent if its solr home is just solr-solrj? this is how the other ones work, e.g. solr-dih, solr-extraction, ... Unlike all of these, however, solrj/src/test-files/ has files that are at the top level, not under its solr home. I was wrong: Solr contribs clustering, dih-extras, extraction, and uima all have files at the top level not under */src/test-files/solr-*/ . These appear mostly to have different names, but in at least one case ( solr-word.pdf ), there is a same-named file in more than one module (extraction and dih-extras). So it would appear that the systemic problem is not fixed by just renaming the solr homes?
        Hide
        Robert Muir added a comment -

        FYI i copied this file here because it used to 'reach' back into extraction to get it...

        Show
        Robert Muir added a comment - FYI i copied this file here because it used to 'reach' back into extraction to get it...
        Hide
        Steve Rowe added a comment -

        Naming solr homes for the contrib in whose test-files dir they reside is an effort at namespacing: allowing each contrib to have its own version of same-named files. It seems to me that leaving files outside of the namespacing directory removes the freedom & protection it affords.

        This is exactly the reason I put everything under solrj/src/test-files/ into a solrj/ sub-directory: several of the top-level files have the same names as those under core/src/test-files/.

        Show
        Steve Rowe added a comment - Naming solr homes for the contrib in whose test-files dir they reside is an effort at namespacing: allowing each contrib to have its own version of same-named files. It seems to me that leaving files outside of the namespacing directory removes the freedom & protection it affords. This is exactly the reason I put everything under solrj/src/test-files/ into a solrj/ sub-directory: several of the top-level files have the same names as those under core/src/test-files/ .
        Hide
        Robert Muir added a comment -

        maybe we should do this for all the modules then?

        Show
        Robert Muir added a comment - maybe we should do this for all the modules then?
        Hide
        Steve Rowe added a comment -

        My vote is to do this for all the modules except Solr core, which can use the anonymous namespace (no namespacing directory under test-files/).

        Show
        Steve Rowe added a comment - My vote is to do this for all the modules except Solr core, which can use the anonymous namespace (no namespacing directory under test-files/ ).
        Hide
        Robert Muir added a comment -

        +1

        Show
        Robert Muir added a comment - +1
        Hide
        Steve Rowe added a comment -

        Patch implementing changes, including modifications to the SOLR-2452 patch hacking script under dev-tools/scripts/.

        Run the attached svn movement script before applying the patch.

        Show
        Steve Rowe added a comment - Patch implementing changes, including modifications to the SOLR-2452 patch hacking script under dev-tools/scripts/. Run the attached svn movement script before applying the patch.
        Hide
        Steve Rowe added a comment -

        Committed:

        • r1149600: trunk
        • r1149601: branch_3x
        Show
        Steve Rowe added a comment - Committed: r1149600: trunk r1149601: branch_3x
        Hide
        Steve Rowe added a comment - - edited

        When running Solrj's tests, ExternalPaths.determineSourceHome() fails, e.g.:

        Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/9695/
        
        15 tests failed.
        FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.SolrExampleBinaryTest
        
        Error Message:
        null
        
        Stack Trace:
        java.lang.ExceptionInInitializerError
        	at org.apache.solr.client.solrj.SolrExampleBinaryTest.beforeTest(SolrExampleBinaryTest.java:36)
        Caused by: java.lang.RuntimeException: Cannot find resource: solr/conf
        	at org.apache.solr.SolrTestCaseJ4.getFile(SolrTestCaseJ4.java:1101)
        	at org.apache.solr.util.ExternalPaths.determineSourceHome(ExternalPaths.java:40)
        	at org.apache.solr.util.ExternalPaths.<clinit>(ExternalPaths.java:30)
        
        Show
        Steve Rowe added a comment - - edited When running Solrj's tests, ExternalPaths.determineSourceHome() fails, e.g.: Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/9695/ 15 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.SolrExampleBinaryTest Error Message: null Stack Trace: java.lang.ExceptionInInitializerError at org.apache.solr.client.solrj.SolrExampleBinaryTest.beforeTest(SolrExampleBinaryTest.java:36) Caused by: java.lang.RuntimeException: Cannot find resource: solr/conf at org.apache.solr.SolrTestCaseJ4.getFile(SolrTestCaseJ4.java:1101) at org.apache.solr.util.ExternalPaths.determineSourceHome(ExternalPaths.java:40) at org.apache.solr.util.ExternalPaths.<clinit>(ExternalPaths.java:30)
        Hide
        Steve Rowe added a comment -

        Patch that allows tests to pass.

        Committing shortly

        Show
        Steve Rowe added a comment - Patch that allows tests to pass. Committing shortly
        Hide
        Steve Rowe added a comment -

        Committed the part2 patch:

        • r1149682: trunk
        • r1149691: branch_3x

        I'll leave the issue open until Jenkins builds including Solr on both trunk and branch_3x have succeeded. (I triggered Solr-3.x and Solr-trunk builds on Jenkins, so successful builds should be finished within a couple of hours.)

        Show
        Steve Rowe added a comment - Committed the part2 patch: r1149682: trunk r1149691: branch_3x I'll leave the issue open until Jenkins builds including Solr on both trunk and branch_3x have succeeded. (I triggered Solr-3.x and Solr-trunk builds on Jenkins, so successful builds should be finished within a couple of hours.)
        Hide
        Steve Rowe added a comment -

        The Solr-3.x and Solr-trunk Jenkins builds have both succeeded.

        Show
        Steve Rowe added a comment - The Solr-3.x and Solr-trunk Jenkins builds have both succeeded.
        Hide
        Robert Muir added a comment -

        bulk close for 3.4

        Show
        Robert Muir added a comment - bulk close for 3.4

          People

          • Assignee:
            Steve Rowe
            Reporter:
            Steve Rowe
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development