Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-10486

Avoid unnecessary overhead in TopScoreDoc and TopField collector manager

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • None
    • None
    • New

    Description

      TopScoreDocCollector and TopFieldCollector expose a createSharedManager method that returns a collector manager for concurrent search, which relies on a shared global counter for hits counting as well as a shared max score accumulator.

      As part of LUCENE-10002 we are going to deprecate the ability to search providing a collector, in favour of using a corresponding collector manager. The above mentioned shared collector managers are great for concurrent searches, yet they add overhead when search is single threaded, which can be the case despite a collector manager is used. That is the case when an executor is not set to the index searcher, or when there's only one slice to be searched.

      We could adapt the hits threshold checker as well as the max score accumulator depending on whether a search is effectively executed by multiple threads or not.

      An additional idea along the same lines could be to introduce a new hits threshold checker for the case when totalHitsThreshold is set to Integer.MAX_VALUE, which does no counting at all. This could be safely used both in the single threaded as well as in the concurrent scenario.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              lucacavanna Luca Cavanna
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m