[LUCENE-1024] MarkovChainTokenFilter

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Component/s: core/termvectors, modules/analysis
• Lucene Fields: New, Patch Available
• Assignee: Unassigned
• Reporter: Karl Wettin
• Created: 09/Oct/07 16:38
• Updated: 28/Aug/22 11:41
• Resolved: 10/Oct/07 09:23

Description

Merges chains of tokens to a single token, ngrams on a phrase word level rathar than on a word character level.

Quite useful for text mining purposes of the vector space.