[TIKA-357] Increase buffer size for meta tag sniffing

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.5
• Fix Version/s: 0.6
• Assignee: Chris A. Mattmann
• Reporter: Kenneth William Krugler
• Created: 21/Dec/09 22:30
• Updated: 20/Jan/10 05:50
• Resolved: 20/Jan/10 05:50

Description

Some web pages (such as makler.su, see attached) have lots of script data before the body of the HTML.

When this happens, the sniffing code fails to find the charset info in the meta tag, because it currently only sniffs the first 4K.

Bumping it to 8K would cover all of the cases that I (Ken) have seen during a test crawl.

Issue Links

• is related to: TIKA-366 Increase buffer size for mime type sniffing
• is related to: TIKA-367 Mime type rootXML equality improvement