[SOLR-3109] group=true requests result in numerous redundant shard requests

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.5, 4.0-ALPHA
• Fix Version/s: 3.6, 4.0-ALPHA
• Component/s: search
• Labels: patch performance

Environment

64-bit Linux, sharded environment

• Assignee: Martijn van Groningen
• Reporter: Russell Black
• Created: 08/Feb/12 05:26
• Updated: 10/May/13 10:39
• Resolved: 13/Feb/12 13:09

Description

During the second phase of a group query, the collator sends a query to each of the shards. The purpose of this query is for shards to respond with the doc ids that match the set of group ids returned from the first phase. The problem is that it sends this second query to each shard multiple times. Specifically, in an environment with n shards, each shard will be hit with an identical query n times during the second phase of query processing, resulting in O(n ²) performance where n is the number of shards.

I have traced this bug down to a single line in TopGroupsShardRequestFactory.java, and I am attaching a patch.