[LUCENE-4220] Replace benchmarks crazy HTML parser by a nekohtml 10-liner

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0-ALPHA
• Fix Version/s: 4.0-BETA, 6.0
• Component/s: modules/benchmark
• Lucene Fields: New
• Assignee: Uwe Schindler
• Reporter: Uwe Schindler
• Created: 14/Jul/12 12:59
• Updated: 28/Aug/22 13:21
• Resolved: 15/Jul/12 17:38

Description

Benchmark contains a javacc-based HTML parser which of course violates all specs, is huge and error prone.

I can replace it by a NEKOHTML based one (approx 10 - 20 lines of code). NEKOHTML is an extension for XERCES (that we already use to read wikipedia), that produces SAX-events or DOM tree out of a HTML file usingg standard XML APIS. We could also use TIKA, but I refuse to download the Internet to get TIKA running for just parsing a HTML file.

Issue Links

• is related to: LUCENE-4589 Upgrade benchmark modules nekohtml and remove turkish HTML element lowercasing workaround!