[LUCENE-185] [PATCH] Thai Analysis Enhancement

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Component/s: modules/analysis

Environment

Operating System: All
Platform: All

• Bugzilla Id: 27182
• Assignee: Unassigned
• Reporter: Pichai Ongvasith
• Created: 24/Feb/04 14:55
• Updated: 28/Aug/22 11:15
• Resolved: 02/Jan/08 14:29

Description

Unlike other languages, Thai do not have a clear word boundary within a
sentence. Words are written consecutively without a delimiter. The Lucene
StandardTokenizer currently cannot tokenize a Thai sentence and returns the
whole sentence as a token. A special tokenizer to break Thai sentences into
words is required.