[TIKA-1602] Detecting standards-non-compliant emails as message/rfc822

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.10
• Component/s: mime
• Flags: Patch
• External issue URL: https://github.com/apache/tika/pull/40
• Assignee: Chris A. Mattmann
• Reporter: Jeremy B. Merrill
• Created: 10/Apr/15 20:03
• Updated: 10/Jul/15 17:59
• Resolved: 01/Jul/15 13:22

Description

Tika does not properly detect certain emails as `message/rfc822` if they're slightly standards-non-compliant and begin with `Status: ` as the first header. I've added `Status: ` as a magic detection line in tika-mimetypes.xml.

This solves my problem and does not appear to cause unit test failures. I have not yet run the tika-batch tests.

As further information, the emails that are processed incorrectly come from dumps directly from various US public officials' mailservers. The dumps, I believe since they're not intended to be transmitted over the wire, sometimes are slightly non-compliant.

It's important to note that Tika (and the underlying library, James Mime4J) do properly parse these emails, despite the non-compliant header. The problem is getting Tika to detect the file as an email so that Mime4J gets chosen to parse it.

Pull request on Github at https://github.com/apache/tika/pull/40

Issue Links

• duplicates: TIKA-879 Detection problem: message/rfc822 file is detected as text/plain.