[SOLR-13336] solrconfig.xml maxBooleanClauses ignored by programtic/rewrtten queries; can result in exponential expansion of naive queries

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 7.0, 8.0
• Fix Version/s: 8.1, 9.0
• Component/s: query parsers
• Assignee: Chris M. Hostetter
• Reporter: Michael Gibney
• Created: 20/Mar/19 01:40
• Updated: 16/Jun/22 12:58
• Resolved: 15/Apr/19 22:03

Description

changes made in Solr 7.0 set the effective value of BoleanQuery.getMaxClauseCount to Integer.MAX_VALUE-1 and only impossed a restriction based on the (existing) solrconfig.xml setting at the Solr query parser level via a new utility helper method.l

But this means programatically generated queries (either by low level lucene methods, or by query re-writing) no longer had any safety valve to prevent (effectively) infinite expansion. This issue fixes this problem by:

• restoring a default upper bound on BoleanQuery.getMaxClauseCount of 1024
• introducing a new solr.xml level setting for configuring this upper bound:

  <int name="maxBooleanClauses">${solr.max.booleanClauses:1024}</int>
  

NOTE that this solr.xml limit is ahard upper bound, that superceeds the existing solrconfig.xml setting, which has been left in place and still limits the size of user specified boolean queries. ie: solr.xml maxBooleanClauses >= solrconfig.xml maxBooleanClauses >= number of clauses a user explicitly specifies in a query string; solr.xml maxBooleanClauses >= numberr of clauses in an expanded/rewritten query

original bug report

Since SOLR-10921 it appears that Solr always sets BooleanQuery.maxClauseCount (at the Lucene level) to Integer.MAX_VALUE-1. I assume this is because Solr parses maxBooleanClauses out of the config and applies it externally.

In any case, when used as part of lucene.util.QueryBuilder.analyzeGraphPhrase (and possibly other places?), the Lucene code checks internally against only the static maxClauseCount variable (permanently set to Integer.MAX_VALUE-1 in the context of Solr).

Thus in at least one case (analyzeGraphPhrase(), but possibly others?), maxBooleanClauses is having no effect. I'm pretty sure this is what's underlying the issue reported here as being related to Solr 7.6.

To summarize, users are definitely susceptible (to varying degrees of likely severity, assuming no actual malicious attack) if:

1. Running Solr >= 7.6.0
2. Using edismax with "ps" param set to >0
3. Query-time analysis chain is at all capable of producing graphs (e.g., WordDelimiterGraphFilter, SynonymGraphFilter that has corresponding synonyms with varying token lengths.

Users are particularly vulnerable in practice if they have query-time WordDelimiterGraphFilter configured with preserveOriginal=true.

To clarify, Lucene/Solr 7.6 didn't exactly introduce the issue; it only increased the likelihood of problems manifesting (as a result of LUCENE-8531). Notably, the "enumerated strings" approach to graph phrase query (reintroduced by LUCENE-8531) was previously in place pre-6.5 – at which point it could rely on default Lucene-level maxClauseCount failsafe (removed as of 7.0). This explains the odd "Affects versions" => maxBooleanClauses was disabled at the Lucene level (in Solr contexts) starting with version 7.0, but the change became more likely to manifest problems for users as of 7.6.

Issue Links

• relates to: SOLR-10921 Make boolean query clause limit configurable per-query
• relates to: LUCENE-8531 QueryBuilder hard-codes inOrder=true for generated sloppy span near queries