[LUCENE-7227] TokenStream contract violation: reset()/close() call missing,

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 6.0
• Component/s: core/queryparser
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Lei Zhang
• Created: 18/Apr/16 11:56
• Updated: 28/Aug/22 14:56

Description

public static void highLightClojure(int id, String text, String field) {
try {
Query queryToSearch;
TokenStream tokenStream = TokenSources.getTokenStream( field,text, analyzer);
queryToSearch = new QueryParser("asddf", analyzer).parse("read text file string utf8");
Highlighter highlighter = new Highlighter(new SimpleHTMLFormatter(),
new QueryScorer(queryToSearch));

TextFragment[] frag = highlighter.getBestTextFragments(tokenStream, text, false, 4);
for (int j = 0; j < frag.length; j++) {
if ((frag[j] != null))

{ System.out.println("score: " + frag[j].getScore() + ", frag: " + (frag[j].toString())); }

}
} catch (IOException e)

{ // TODO Auto-generated catch block e.printStackTrace(); } catch (InvalidTokenOffsetsException e) { // TODO Auto-generated catch block e.printStackTrace(); }

catch (ParseException e)

{ // TODO Auto-generated catch block e.printStackTrace(); }

}

Exception:
TokenStream contract violation: reset()/close() call missing,
reset() called multiple times, or subclass does not call
super.reset(). Please see Javadocs of TokenStream class for more
information about the correct consuming workflow.

The tokenStream must be created after QueryParser, its an implicit dependency, but no any information mentioned it.