[LUCENE-6200] Highlighter sometime went wrong

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 4.10.2
• Component/s: modules/highlighter
• Labels: highlighter
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: thihy
• Created: 26/Jan/15 15:45
• Updated: 5 days ago 21:12
• Resolved: 29/May/15 02:00

Description

I have write a test case for this. I expect "<B>游戏</B>是<B>游戏</B>", but get "<B>游戏是游戏</B>"

	public static void main(String[] args) throws IOException,
			InvalidTokenOffsetsException {
		String text = "游戏是游戏";
		String query = "游戏";

		CJKAnalyzer analyzer = new CJKAnalyzer();
		Scorer fragmentScorer = new QueryScorer(new TermQuery(new Term("field",
				query)));
		Highlighter highlighter = new Highlighter(fragmentScorer);
		String fragment = highlighter.getBestFragment(
				analyzer.tokenStream("field", text), text);
		analyzer.close();
		System.out.println(fragment); // println: <B>游戏是游戏</B>
	}

Issue Links

• duplicates: LUCENE-1489 highlighter problem with n-gram tokens