[SOLR-15579] SQL WHERE filter with 20 or more values in the IN clause results in no filtering on that field

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.0, 8.10
• Component/s: Parallel SQL
• Labels: RobustSQL
• Assignee: Timothy Potter
• Reporter: Timothy Potter
• Created: 06/Aug/21 21:04
• Updated: 28/Sep/21 20:27
• Resolved: 12/Aug/21 16:44

Description

Calcite has a default threshold of 20 for the number of values in an IN clause. If this threshold is reached, then Calcite transforms the IN clause into an inline table + sub-query which is not handled by SolrFilter, so that filter clause gets dropped from the resulting query to Solr, see:

https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L1143

The easiest solution is to update the config.withInSubQueryThreshold(Integer.MAX_VALUE)

I think we should also explore how to work with the sub-query against the inline table if possible.

Issue Links

• links to: GitHub Pull Request #249
• links to: GitHub Pull Request #2553