[LUCENE-9648] Add a numbers query in sandbox that takes advantage of index sorting.

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Won't Do
• Component/s: modules/sandbox
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Feng Guo
• Created: 27/Dec/20 08:55
• Updated: 28/Aug/22 16:11
• Resolved: 13/Dec/21 10:46

Description

LUCENE-6539 introduced DocValuesNumbersQuery to benefit some special cases (many terms/numbers and fewish matching hits). Inspired by LUCENE-7714, maybe we can speed it up when meeting index sort since we can do a binary search to bound the docvalues before iter it.

Here are some simple benchmarks:

Based on 50,000,000 doc, each long in query can hit at least one doc:

LongsCount	PointInSetQuery(s/op)	IndexSortDocValuesNumbersQuery(s/op)	DocValuesNumbersQuery(s/op)
1000	0.144 ± 0.001	0.428 ± 0.008	1.159 ± 0.031
5000	0.404 ± 0.003	0.424 ± 0.008	1.343 ± 0.022
10000	0.661 ± 0.012	0.436 ± 0.008	1.372 ± 0.011
50000	1.319 ± 0.011	0.450 ± 0.005	1.173 ± 0.017
100000	1.328 ± 0.007	0.459 ± 0.003	1.160 ± 0.098

Based on 100,000,000 doc, each long in query can hit at least one doc:

LongsCount	PointInSetQuery(s/op)	IndexSortDocValuesNumbersQuery(s/op)	DocValuesNumbersQuery(s/op)
1000	0.189 ± 0.003	0.860 ± 0.028	2.408 ± 0.087
5000	0.715 ± 0.010	0.879 ± 0.010	2.642 ± 0.105
10000	1.075 ± 0.034	0.859 ± 0.010	2.629 ± 0.077
50000	2.652 ± 0.114	0.865 ± 0.009	2.737 ± 0.099
100000	2.905 ± 0.027	0.881 ± 0.005	2.820 ± 0.474

Issue Links

• links to: GitHub Pull Request #2170