[LUCENE-9426] UnifiedHighlighter ANALYSIS mode does not accurately highlight SpanNotQuery or MUST_NOT

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 8.5.1
• Component/s: modules/highlighter
• Labels: easyfix

Environment

I tested with 8.5.1, but other versions are probably also affected.

• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Christoph Goller
• Created: 14/Jul/20 10:46
• Updated: 28/Aug/22 16:04

Description

If UnifiedHighlighter uses MemoryIndexOffsetStrategy, it does not treat SpanNotQuery correctly.
Since UnifiedHighlighter uses actual search in order to determine which locations to highlight, it should be consistent with search and only highlight locations in a document that really match the query. However, it does not for SpanNotQuery.

For the query spanNot(spanNear([content:100, content:dollars], 1, true), content:thousand, 0, 0)
it produces
A <b>100</b> fucking <b>dollars</b> wasn't enough to fix it. ... We need <b>100</b> thousand <b>dollars</b> to buy the house