[LUCENE-3888] split off the spell check word and surface form in spell check dictionary

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Fix Version/s: 4.9, 6.0
• Component/s: modules/spellchecker
• Lucene Fields: New
• Assignee: Koji Sekiguchi
• Reporter: Koji Sekiguchi
• Created: 20/Mar/12 07:08
• Updated: 28/Aug/22 13:11

Description

The "did you mean?" feature by using Lucene's spell checker cannot work well for Japanese environment unfortunately and is the longstanding problem, because the logic needs comparatively long text to check spells, but for some languages (e.g. Japanese), most words are too short to use the spell checker.

I think, for at least Japanese, the things can be improved if we split off the spell check word and surface form in the spell check dictionary. Then we can use ReadingAttribute for spell checking but CharTermAttribute for suggesting, for example.