Description
This is a proposal to substantially refactor SolrTestCaseJ4 and some of its intermediate subclasses in the hierarchy. In essence, I envision that tests should work with a SolrClient typed "solrClient" field managed by the test infrastructure. With only a few lines of code, a test should be able to pick between an instance based on EmbeddedSolrServer (lighter tests), HttpSolrClient (tests HTTP/Jetty behavior directly or indirectly), SolrCloud, and perhaps a special one for our distributed search tests. STCJ4 would refactor its methods to use the solrClient field instead of TestHarness. TestHarness would disappear as-such; bits of its existing code would migrate elsewhere, such as to manage an EmbeddedSolrServer for testing.
I think we can do a transition like this in stages and furthermore minimally affecting most tests by adding some deprecated shims. Perhaps STCJ4 should become the deprecated shim so that users can still use it during 7.x and to help us with the transition internally too. More specifically, we'd add a new superclass to STCJ4 that is the future – "SolrTestCase".
Additionally, there are a bunch of methods on SolrTestCaseJ4 that I question the design of, especially ones that return XML strings like delI (generates a delete-by-id XML string) and adoc. Perhaps that used to be a fine idea before there was a convenient SolrClient API but we've got one now and a test shouldn't be building XML unless it's trying to test exactly that.
For consulting work I once developed a JUnit4 TestRule managing a SolrClient that is declared in a test with an annotation of @ClassRule. I had a variation for SolrCloud and EmbeddedSolrServer that was easy for a test to choose. Since TestRule is an interface, I was able to make a special delegating SolrClient subclass that implements TestRule. This isn't essential but makes use of it easier since otherwise you'd be forced to call something like getSolrClient(). We could go the TestRule route here, which I prefer (with or without having it subclass SolrClient), or we could alternatively do TestCase subclassing to manage the lifecycle.
Initially I'm just looking for agreement and refinement of the approach. After that, sub-tasks ought to be added. I won't have time to work on this for some time.
Attachments
Issue Links
- is a parent of
-
SOLR-13473 Ensure that the tests use JSON,javabin to update docs
- Open
- is related to
-
SOLR-12801 Fix the tests, remove BadApples and AwaitsFix annotations, improve env for test development.
- Open
- mentioned in
Refactor and document... How to use the new refactored version should be covered in the Ref Guide. Presently I wind up reading other tests to find out how things work, and very often copy/paste/modify other tests to get going. It would be nice if one could refer to documentation that pointed out the options available and gave some advice on when to use what.
Yes, Read/Write of any writer type should be thoroughly tested in a single place and then presumed to work elsewhere, so it shouldn't matter which is used by the test (could be randomized?).
Also I think it should ideally be easy to flip between:
How ever we do things, it has to be easy for the test writer to see/understand that they HAVE exercised both API's, and to get printouts of exactly what was sent as a post-body, headers, request line etc... For testing API command implementations it's pretty key to make sure all the bases are covered, and that the v1/v2 conversions and property substitutions are happening properly behind the scenes. Other usages possibly don't care as much... which is something to think about too.
Also, it seems that there are about 30 tests that turn off SSL to avoid random keystore errors... that also needs to be fixed, though probably a separate issue.