Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-3109

group=true requests result in numerous redundant shard requests

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.5, 4.0-ALPHA
    • Fix Version/s: 3.6, 4.0-ALPHA
    • Component/s: search
    • Labels:
    • Environment:

      64-bit Linux, sharded environment

      Description

      During the second phase of a group query, the collator sends a query to each of the shards. The purpose of this query is for shards to respond with the doc ids that match the set of group ids returned from the first phase. The problem is that it sends this second query to each shard multiple times. Specifically, in an environment with n shards, each shard will be hit with an identical query n times during the second phase of query processing, resulting in O(n 2) performance where n is the number of shards.

      I have traced this bug down to a single line in TopGroupsShardRequestFactory.java, and I am attaching a patch.

        Attachments

        1. SOLR-3109.patch
          11 kB
          Russell Black
        2. SOLR-3109-lucene_solr_3_5.patch
          10 kB
          Russell Black
        3. SOLR-3109-Backport-of-grouping-performace-fix-to-3.x.patch
          12 kB
          Greg Bowyer
        4. SOLR-3109.patch
          8 kB
          Martijn van Groningen
        5. SOLR-3109.patch
          1 kB
          Russell Black

          Activity

            People

            • Assignee:
              martijn.v.groningen Martijn van Groningen
              Reporter:
              rblack Russell Black
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: