[OPENNLP-568] Doccat command line tagger should assume whitespace tokenized input

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: tools-1.5.2-incubating
• Fix Version/s: tools-1.5.3
• Component/s: Command Line Interface, Doccat
• Assignee: Jörn Kottmann
• Reporter: Jörn Kottmann
• Created: 02/Apr/13 20:39
• Updated: 02/Apr/13 20:46
• Resolved: 02/Apr/13 20:46

Description

The DoccatTool should read the doccat default format from stdin. The default format is whitespace tokenized, but the DoccatTool uses the Simple Tokenizer to tokenize the input text.

To fix this issue use the Whitespace Tokenizer instead of the Simple Tokenizer.