[SOLR-3222] Pull optimal cache warming queries from a warm solr instance

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 3.5, 4.0-ALPHA
• Component/s: search
• Labels: patch performance
• Assignee: Unassigned
• Reporter: Russell Black
• Created: 09/Mar/12 01:33
• Updated: 15/Mar/12 22:04
• Resolved: 15/Mar/12 22:04

Description

Ever wondered what queries to use to prime your cache? This patch allows you to query a warm running instance for a list of warming queries. The list is generated from the server's caches, meaning you get back an optimal set of queries. The set is optimal to the extent that the caches are optimized. The queries are returned in a format that can be consumed by the

<listener event="firstSearcher" class="solr.QuerySenderListener">

section of solrconfig.xml.

One can use this feature to generate a static set of good warming queries to place in solrconfig.xml under

<listener event="firstSearcher" class="solr.QuerySenderListener">

It can even be used in a dynamic fashion like this:

<listener event="firstSearcher" class="solr.QuerySenderListener">
  <xi:include href="http://host/solr/core/autowarm" xpointer="element(/1/2)" xmlns:xi="http://www.w3.org/2001/XInclude"/>
</listener>

which can work well in certain distributed load-balanced architectures, although in production it would be wise to add an <xi:fallback> element to the include in the event that the host is down.

I implemented this by introducing a new request handler:

  <requestHandler name="/autowarm" class="solr.AutoWarmRequestHandler" />

The request handler pulls a configurable number of "top" keys from the filterCache,fieldValueCache, and queryResultCache. For each key, it constructs a query that will cause that key to be placed in the associated cache. The list of constructed queries are then returned in the response.

Patch to follow.