[LUCENE-7279] AIOOBE from JapaneseTokenizer

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.5.2, 5.6, 6.0.1, 7.0
• Component/s: modules/analysis
• Lucene Fields: New
• Assignee: Michael McCandless
• Reporter: Michael McCandless
• Created: 11/May/16 18:40
• Updated: 28/Aug/22 14:57
• Resolved: 15/Jun/16 21:23

Description

On certain Japanese input strings you can hit this:

java.lang.ArrayIndexOutOfBoundsException: -1
	at __randomizedtesting.SeedInfo.seed([C6752A567B924B1:2B195610610ED60]:0)
	at org.apache.lucene.analysis.ja.JapaneseTokenizer.backtrace(JapaneseTokenizer.java:1607)
	at org.apache.lucene.analysis.ja.JapaneseTokenizer.parse(JapaneseTokenizer.java:902)
	at org.apache.lucene.analysis.ja.JapaneseTokenizer.incrementToken(JapaneseTokenizer.java:479)
	at org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testBigDocument(TestJapaneseTokenizer.java:837)

I have a patch with a test case and fix.