Details
-
Bug
-
Status: Closed
-
Trivial
-
Resolution: Fixed
-
None
-
None
-
Operating System: All
Platform: All
-
32687
Description
Apparently, in ChineseTokenizer, offset should be decremented like bufferIndex
when Character is OTHER_LETTER. This directly affects startOffset and endOffset
values.
This is critical to have Highlighter working correctly because Highlighter marks
matching text based on these offset values.