[LUCENE-8875] Should TopScoreDocCollector Always Populate Sentinel Values?

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.2
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Atri Sharma
• Created: 23/Jun/19 21:37
• Updated: 28/Aug/22 15:47
• Resolved: 10/Jul/19 17:41

Description

TopScoreDocCollector always initializes HitQueue as the PQ implementation, and instruct HitQueue to populate with sentinels. While this is a great safety mechanism, for very large datasets where the query's selectivity is high, the sentinel population can be redundant and can become a large enough bottleneck in itself. Does it make sense to introduce a new parameter in TopScoreDocCollector which uses a heuristic (say number of hits > 10k) and does not populate sentinels?

Issue Links

• is related to: LUCENE-10302 PriorityQueue: optimize where we collect then iterate by using O(N) heapify
• links to: GitHub Pull Request #754