[LUCENE-5460] Allow driving a query by sparse filters

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: core/search
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Shai Erera
• Created: 20/Feb/14 13:15
• Updated: 28/Aug/22 14:00
• Resolved: 16/Feb/15 18:19

Description

Today if a filter is very sparse we execute the query in sort of a leap-frog manner between the query and filter. If the query is very expensive to compute, and/or matching few docs only too, calling scorer.advance(doc) just to discover the doc it landed on isn't accepted by the filter, is a waste of time. Since Filter is always the "final ruler", I wonder if we had something like boolean DISI.advanceExact(doc) we could use it instead, in some cases.

There are many combinations in which I think we'd want to use/not-use this API, and they depend on: Filter's complexity, Filter.cost(), Scorer.cost(), query complexity (span-near, many clauses) etc.

I open an issue so we can discuss. DISI.advanceExact(doc) is just a preliminary proposal, to get an API we could experiment with. The default implementation should be fairly easy and straightforward, and we could override where we can offer a more optimized imp.

Issue Links

• is related to: LUCENE-5424 FilteredQuery useRandomAccess() should use cost()
• is superceded by: LUCENE-6198 two phase intersection