[MAHOUT-1608] Add Option WikipediaToSequenceFile to remove Category Labels from Documents

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.9
• Fix Version/s: 0.10.0
• Assignee: Andrew Palumbo
• Reporter: Andrew Palumbo
• Created: 23/Aug/14 03:01
• Updated: 13/Apr/15 10:19
• Resolved: 29/Aug/14 00:16

Description

Currently WikipediaMapper job extracts Category labels from the text of the Wikipedia documents and leaves the label as [[Category:label]] in the document. Add in an option to WikipediaToSequenceFile.java to remove [[Category:label]] from the text after extracting the label.