[MAHOUT-1564] Naive Bayes Classifier for New Text Documents

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 0.9
• Fix Version/s: 0.10.0
• Labels: DSL legacy scala spark
• Assignee: Andrew Palumbo
• Reporter: Andrew Palumbo
• Created: 27/May/14 21:30
• Updated: 30/May/15 17:50
• Resolved: 01/Apr/15 21:25

Description

MapReduce and DSL Naive Bayes implementations currently lack the ability to classify a new document (outside of the training/holdout corpus). This New feature will do the following.

1. Vectorize a new text document using the dictionary and document frequencies from the training/holdout corpus

• assume the original corpus was vectorized using `seq2sparse`; step (1) will use all of the same parameters.

2. Score and label a new document using a previously trained model.

This effort will need to be done in parallel for MRLegacy and DSL implementations. Neither should be too much work.

Issue Links

• relates to: MAHOUT-1493 Port Naive Bayes to the Spark DSL