[TIKA-2592] HTML with charset unicode handled as utf-16 instead utf-8

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.16, 1.17, 2.0.0
• Fix Version/s: 1.18, 2.0.0
• Assignee: Unassigned
• Reporter: Andreas Meier
• Created: 28/Feb/18 08:14
• Updated: 12/Apr/21 13:00
• Resolved: 07/Mar/18 20:14

Description

HTML files are detected as utf-16 when meta content is set to "unicode".

<meta http-equiv="Content-Type" content="text/html; charset="unicode">
 

 
Shouldn't the default be utf-8?

The attached sample file is shown correctly in:
Chromium Version 55.0.2883.75
Firefox 50.1.0
IE 11

I am aware that there is no charset "unicode" (available character encodings: http://www.iana.org/assignments/character-sets/character-sets.xhtml)

Unfortunately there are many wrong encodings used out there.

All unknown encodings should be validated or at least be set to default utf-8.

Regards

Andreas

Issue Links

• causes: TIKA-2758 Possible error charset detection