As part of writing tests for
SOLR-1657, I rewrote one of the base classes (BaseTokenTestCase) to use the new TokenStream API, but also with some additional safety.
Setting the term text to bogus values helps find bugs in tokenstreams that do not clear or clone properly. In this case there is a problem with a tokenstream AB_AAB_Stream in TestBufferedTokenStream, it converts A B -> A A B but does not clone, so the values get overwritten.
This can be fixed in two ways:
- BufferedTokenStream does the cloning
- subclasses are responsible for the cloning
The question is which one should it be?