[TIKA-711] Word parser doesn't extract optional hyphen correctly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0
• Component/s: parser
• Assignee: Michael McCandless
• Reporter: Michael McCandless
• Created: 09/Sep/11 16:40
• Updated: 03/Oct/11 18:26
• Resolved: 03/Oct/11 18:26

Description

We seem not to extract the optional hyphen character correctly in
the Word parser.

You can create this char in Word by typing ctrl and -. It's hidden,
normally; you have to turn on display of formatting marks to see it.

Ideally we'd get U+00AD (unicode soft hyphen), I think.

DOC produces a unicode replacement char, which is wrong.

DOCX and PDF drop the char (which seems acceptable). RTF produces
U+2027 (hyphenation point) which also seems OK (in TIKA-683 it will
produce U+00AD).

PPT and PPTX work correctly (U+00AD).

So DOC is the only bug I think – I haven't dug into what's wrong
yet...