[PIG-3197] Add the Stanford Tokenizer to the list of Tokenizers in Pig: StanfordTokenize

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: internal-udfs
• Assignee: Russell Jurney
• Reporter: Russell Jurney
• Created: 18/Feb/13 23:53
• Updated: 22/Feb/13 14:53

Description

See https://github.com/Ganglion/varaha/blob/master/src/main/java/varaha/text/StanfordTokenize.java

More about the stanford tokenizer here: http://nlp.stanford.edu/software/tokenizer.shtml