[OPENNLP-861] Add Chi-Squared Data Indexer for Feature Selection

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Abandoned
• Affects Version/s: 1.6.0
• Component/s: Machine Learning
• Labels: features
• Assignee: Unassigned
• Reporter: Joey Hong
• Created: 25/Aug/16 20:33
• Updated: 22/Dec/23 10:23
• Resolved: 22/Dec/23 10:23

Description

Text classification will naturally produce a lot of features. A lot of them are independent of the category, and provide no real information gain in the classification.

The Chi-Squared feature selection method will allow features that do not pass a threshold for dependency to be removed from the feature list, keeping the feature list a reasonable size without significantly affecting the classification accuracy.