[LUCENE-10477] SpanBoostQuery.rewrite was incomplete for boost==1 factor

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 8.11.1
• Fix Version/s: 10.0 (main), 8.11.2, 9.2
• Lucene Fields: New
• Assignee: Christine Poerschke
• Reporter: Christine Poerschke
• Created: 21/Mar/22 12:59
• Updated: 10/Dec/23 09:21
• Resolved: 18/May/22 10:58

Description

(This bug report concerns pre-9.0 code only but it's so subtle that it warrants sharing I think and maybe fixing if there was to be a 8.11.2 release in future.)

Some existing code e.g. https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.11.1/lucene/queryparser/src/java/org/apache/lucene/queryparser/xml/builders/SpanNearBuilder.java#L54 adds a SpanBoostQuery even if there is no boost or the boost factor is 1.0 i.e. technically wrapping is unnecessary.

Query rewriting should counteract this somewhat except it might not e.g. note at https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.11.1/lucene/core/src/java/org/apache/lucene/search/spans/SpanBoostQuery.java#L81-L83 how the rewrite is a no-op i.e. this.query.rewrite is not called!

This can then manifest in strange ways e.g. during highlighting:

...
java.lang.IllegalArgumentException: Rewrite first!
	at org.apache.lucene.search.spans.SpanMultiTermQueryWrapper.createWeight(SpanMultiTermQueryWrapper.java:99)
	at org.apache.lucene.search.spans.SpanNearQuery.createWeight(SpanNearQuery.java:183)
	at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extractWeightedSpanTerms(WeightedSpanTermExtractor.java:295)
	...

This stacktrace is not from 8.11.1 code but the general logic is that at line 293 rewrite was called (except it didn't a full rewrite because of SpanBoostQuery wrapping around the SpanNearQuery) and so then at line 295 the IllegalArgumentException("Rewrite first!") arises: https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.11.1/lucene/core/src/java/org/apache/lucene/search/spans/SpanMultiTermQueryWrapper.java#L101

Issue Links

• is related to: LUCENE-10464 unnecessary for-loop in WeightedSpanTermExtractor.extractWeightedSpanTerms
• relates to: SOLR-16112 DefaultSolrHighlighter.doHighlighting to rewrite sufficiently
• relates to: LUCENE-8637 WeightedSpanTermExtractor unnexessarily enforces rewrite for some SpanQueiries
• links to: GitHub Pull Request #758
• links to: GitHub Pull Request #2650
• links to: https://github.com/apache/lucene/pull/737