[LUCENE-6903] Enhance WordDelimiterFilter to skip operation on tokens marked as keywords.

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 5.3.1
• Component/s: modules/analysis
• Lucene Fields: New, Patch Available
• Assignee: Shawn Heisey
• Reporter: Shawn Heisey
• Created: 19/Nov/15 18:24
• Updated: 28/Aug/22 14:46

Description

We have a github pull request for a feature to allow WDF to skip operation when the keyword marker filter has marked tokens.

https://github.com/apache/lucene-solr/pull/210

This will also add a new configuration parameter called "splitKeywordTokens" that will default to false in 6.0 and true in earlier versions.