[SOLR-15429] Solr implications of LUCENE-8811 changes to how (solr.xml's) maxBooleanClauses is now used by IndexSearcher

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.0
• Assignee: Chris M. Hostetter
• Reporter: Chris M. Hostetter
• Created: 24/May/21 23:28
• Updated: 12/May/22 00:25
• Resolved: 03/Jun/21 21:27

Description

Background Info

LUCENE-8811 Modified Lucene 9.x to expanded the usage of BooleanQuery.maxClauseCount (now moved to IndexSearcher.maxClauseCount)

Now, in addition to being a limit on the number of individual clauses that can be specified when creating single BooleanQuery (either explicitly or during Query.rewrite()) it is also used by a QueryVisitor (after Query.rewrite() to enforce a limit on the total number of clauses in a nested query structure.

In other words, the following psuedo code will work in lucene 8, but fail in lucene 9...

x = new BooleanQuery()
y = new BooleanQuery()
for (int i = 0; i < BooleanQuery.maxClauseCount - 1; i++) {
  x.addClause(new TermQuery("fx:" + i))
  y.addClause(new TermQuery("fy:" + i))
}
BooleanQuery wrapper = new BooleanQuery(x, y)
searcher.search(wrapper)

(Current) Impacts on Solr users

The solr.xml maxBooleanClauses setting is what currently controls how this value is configured. If unset, it defaults to the lucene default (unchanged in 9.x) of 1024 – but the solr.xml files shipped with all versions of Solr since 8.1 have specified a value of "\${solr.max.booleanClauses:1024}" (see SOLR-13336)

The direct impacts of LUCENE-8811 on Solr users (as of the current solr.git/main branch) is that queries which worked fine in 8.x may start throwing TooManyClauses() exceptions in 9.x, even with existing (built by 8.x) indexes, depending on how complex and deeply nested the query structures are.

This jira is being created as a focal point for assess/discussion what changes we want to make to Solr code and/or documentation regarding maxBooleanClauses in Solr 9.0.

 

Issue Links

• relates to: LUCENE-8811 Add maximum clause count check to IndexSearcher rather than BooleanQuery
• relates to: LUCENE-9970 provide apps details about why TooManyClauses was thrown