[TIKA-2257] Arabic vowel marks displaced when reading from PDF

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.14
• Component/s: parser
• Assignee: Unassigned
• Reporter: Christopher Creutzig
• Created: 30/Jan/17 14:21
• Updated: 02/Feb/17 08:04

Description

The attached file, in its second line, contains “العَرَبِيَّة”. It was created on Mac OS X 10.11.6 by selecting “Save As PDF” from the system print dialog started from Microsoft Word.

Reading the text from the PDF file, the short a vowel marks are displaced, returning
U+0627 U+0644 U+064E U+0639 U+064E U+0631 U+0628 U+0650 U+06CC U+0651 U+064E U+0629 instead of the expected
U+0627 U+0644 U+0639 U+064E U+0631 U+064E U+0628 U+0650 U+064A U+064E U+0651 U+0629 (الَعَربِیَّة instead of العَرَبِيَّة).

Here is the (MATLAB) code used for reading:

handler = org.apache.tika.sax.ToXMLContentHandler;
parser = org.apache.tika.parser.AutoDetectParser;
metadata = org.apache.tika.metadata.Metadata;
fh = java.io.FileInputStream(fullname);
parser.parse(fh, handler, metadata);
s = string(handler.toString);
fh.close;

Issue Links

• depends upon: PDFBOX-3674 Incorrect ordering of fatha -- potentially indicative of larger issue with RTL