[LUCENE-8990] IndexOrDocValuesQuery can take a bad decision for range queries if field has many values per document

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.3
• Lucene Fields: New
• Assignee: Ignacio Vera
• Reporter: Ignacio Vera
• Created: 26/Sep/19 13:04
• Updated: 28/Aug/22 15:51
• Resolved: 04/Oct/19 08:22

Description

Heuristics of IndexOrDocValuesQuery are somewhat inconsistent for range queries . The leadCost that is provided is based on number of documents, meanwhile the cost() of a range query is based on the number of points that potentially match the query. 

Therefore it might happen that a BKD tree has millions of points but this points correspond to just a few documents. Therefore we can take the decision of executing the query using docValues and in fact we are almost scanning all the points.

Maybe the cost() function for range queries need to take into account the average number of points per document in the tree and adjust the value accordingly.

 

Issue Links

• links to: GitHub Pull Request #905