[OPENNLP-205] Refactor the SentenceDetectorME class to do the mapping of end-of-sent positions to spans better

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Sentence Detector
• Assignee: Unassigned
• Reporter: Jörn Kottmann
• Created: 27/Jun/11 08:16
• Updated: 27/Jun/11 08:16

Description

The SentenceDectorME class should be refactored to improve the mapping of end-of-sent positions to spans better. The current code tries to eliminate white spaces between to sentences, but this code fails in case the UseTokenEnd option is set to false. If set to true the sentence detector might not work correctly in all cases.