[TIKA-2396] Unexpected charset detected for a plain text file by CharsetDetector

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: detector, parser
• Assignee: Unassigned
• Reporter: Gaurav Gupta
• Created: 16/Jun/17 10:26
• Updated: 16/Jun/17 12:56

Description

Hi,

The CharsetDetector seems to be incorrectly detecting IBM424_rtl charset with maximum probability for the text file attached - test_Asset.txt . ISO-8859-9 has the second-best confidence value which ideally should have first in the list.
Versions being used:

apache-core - 1.14.0
apache-parsers-1.14.0

Thanks