[TIKA-2737] regression in charset detection

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.17, 1.18, 1.19
• Component/s: detector
• Assignee: Unassigned
• Reporter: rdamir
• Created: 25/Sep/18 17:28
• Updated: 02/Nov/18 20:10

Description

The attached text file is a test csv file (cbp12pr_ia_st.txt) I'm using for testing of csv parser. from version 1.13 to 1.16 - the test was working. I'm trying to upgrade to the latest version 1.19. The test started failing with version 1.17 (see attachments for matches in version 1.16 as well as 1.17). The attached test file contain method testFailure (the last one) that show the wrong detection the expected is UTF-8 detected IBM500.

Issue Links

• is related to: TIKA-2771 enableInputFilter() wrecks charset detection for some short html documents