[LUCENE-9588] Exceptions handling in methods of SegmentingTokenizerBase

• Type: Improvement
• Status: Patch Available
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 8.6.3
• Component/s: modules/analysis
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Nguyen Minh Gia Huy
• Created: 28/Oct/20 08:01
• Updated: 24/Nov/24 04:30

Description

The current interface of setNextSentence and incrementWord methods in SegmentingTokenizerBase do not define the checked exceptions, which makes it troublesome to be inherited.

For example, if we override the incrementWord  with a logic that invoke  incrementToken on another tokenizer, the incrementToken raises the IOException but the incrementWord is not defined to handle it.

I think having setNextSentence and incrementWord handle the IOException would make the SegmentingTokenizerBase easier to be used.