[SOLR-16945] Add multi-threaded warming support to CaffeineCache and QuerySenderListener

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Server
• Assignee: Unassigned
• Reporter: Tim Underwood
• Created: 22/Aug/23 21:04
• Updated: 28/Jan/24 00:00

Description

Cache warming (via CaffeineCache) and newSearcher/firstSearcher listeners (via QuerySenderListener) currently run queries single-threaded. It would be nice to be able to specify running these with multiple threads to speed up the warming process when we know there is extra CPU capacity.

For our use case we are trying to minimize the amount of time it takes for newly indexed changes to be visible to users while also minimizing any response time spikes caused by opening up new searchers and repopulating caches. We currently make use of cache warming and listeners to warm new searchers, however, this process is currently too slow when running single-threaded. We run our servers ~50% loaded and would like to be able to make use of extra CPU capacity to speed up the warming.

I propose adding an optional "autowarmThreads" attribute when specifying the caches. For example:

    <filterCache size="512"
                 initialSize="512"
                 autowarmCount="512"
                 autowarmThreads="4"/>

The QuerySenderListener configuration would use a <threads> element (I didn't see an obvious say to use an attribute there). For example:

    <listener event="firstSearcher" class="solr.QuerySenderListener">
      <threads>4</threads>
      <arr name="queries">
        <!--
        <lst>
          <str name="q">static firstSearcher warming in solrconfig.xml</str>
        </lst>
        -->
      </arr>
    </listener>

In both cases the number of threads is optional. The default behavior is the existing single-threaded approach. Using multiple threads is opt-in.

 

Issue Links

• links to: GitHub Pull Request #1861