[TIKA-1102] Can we add <div> to the list of heuristics for bad html fragments?

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2, 1.3
• Fix Version/s: 1.4
• Component/s: parser

Environment

I'm using Solr 4.0 final with tika v1.2 and ManifoldCF v1.2dev all on tomcat 7.0.37

• Assignee: Kenneth William Krugler
• Reporter: David Morana
• Created: 05/Apr/13 14:18
• Updated: 28/May/13 14:27
• Resolved: 28/May/13 14:27

Description

Good morning,
Crawling legacy sites with poorly written html fragments causes severe Solr Xml parse errors and in turn causes ManifoldCF to abort.
Can we add <div> to the list of heuristics so the html parser is used instead of the xml parser?
see this ticket for further information: TIKA-1101

Thank you,

Issue Links

• relates to: TIKA-1101 XML parse error caused by org.xml.sax.SAXParseException;The entity "nbsp" was referenced, but not declared