[LUCENE-1215] Support of Unicode Collation

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: modules/analysis
• Lucene Fields: New, Patch Available
• Assignee: Unassigned
• Reporter: Hiroaki Kawai
• Created: 11/Mar/08 05:26
• Updated: 28/Aug/22 11:47
• Resolved: 17/Apr/10 14:28

Description

New in java 6, we have java.text.Normalizer that supports Unicode Standard Annex #15 normalization.
http://java.sun.com/javase/6/docs/api/java/text/Normalizer.html
http://www.unicode.org/unicode/reports/tr15/

The normalization defined has four variants of C, D, KC, KD. Canonical Decomposition or Compatibility Decomposition will be normalize the representation of a String, and the search result will be improved.

I'd like to submit a TokenFilter code supporting this feature!