Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
-
New
Description
Investigating a test failure post-LUCENE-8757, I realized that TestTopDocsMerge takes a non-obvious invariant on the fact that the number of Collectors involved in the merge will be equal to the number of LeafReaderContexts originally present. This is propagated in the corresponding ScoreDocs's shardIndex fields, which can lead to subtle issues since shardIndex is used for tie-breaking in the priority queue used during the merge. I believe that this is a dangerous and unnecessary dependency to take since the IndexSearcher#slices method does not advertise any such guarantees.
The underlying assumption worked well in the past since the default slice allocation algorithm always allocated a slice per segment, thus guaranteeing that the number of Collectors (== number of Slices) will be equal to the number of Leaf Contexts. With 8757, this is no longer true.
I propose a rewrite of the test, where ShardSearcher is allowed to take a LeafSlice instance and can internally do a sequential search on the leaf contexts of the passed in the slice. This will allow TestTopDocsMerge to create N subsearchers where N is equal to the number of slices used by the IndexSearcher being compared to.