[SOLR-3524] Make discard-punctuation feature in Kuromoji configurable from JapaneseTokenizerFactory

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.6
• Fix Version/s: 4.0-BETA, 6.0
• Component/s: Schema and Analysis
• Assignee: Christian Moen
• Reporter: Kazuaki Hiraga
• Created: 08/Jun/12 02:56
• Updated: 09/May/16 18:48
• Resolved: 12/Jul/12 10:15

Description

JapaneseTokenizer, Kuromoji doesn't provide configuration option to preserve punctuation in Japanese text, although It has a parameter to change this behavior. JapaneseTokenizerFactory always set third parameter, which controls this behavior, to true to remove punctuation.
I would like to have an option I can configure this behavior by fieldtype definition in schema.xml.