Solr
  1. Solr
  2. SOLR-3109

group=true requests result in numerous redundant shard requests

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 3.5, 4.0-ALPHA
    • Fix Version/s: 3.6, 4.0-ALPHA
    • Component/s: search
    • Labels:
    • Environment:

      64-bit Linux, sharded environment

      Description

      During the second phase of a group query, the collator sends a query to each of the shards. The purpose of this query is for shards to respond with the doc ids that match the set of group ids returned from the first phase. The problem is that it sends this second query to each shard multiple times. Specifically, in an environment with n shards, each shard will be hit with an identical query n times during the second phase of query processing, resulting in O(n 2) performance where n is the number of shards.

      I have traced this bug down to a single line in TopGroupsShardRequestFactory.java, and I am attaching a patch.

      1. SOLR-3109.patch
        11 kB
        Russell Black
      2. SOLR-3109.patch
        8 kB
        Martijn van Groningen
      3. SOLR-3109.patch
        1 kB
        Russell Black
      4. SOLR-3109-Backport-of-grouping-performace-fix-to-3.x.patch
        12 kB
        Greg Bowyer
      5. SOLR-3109-lucene_solr_3_5.patch
        10 kB
        Russell Black

        Activity

        Russell Black created issue -
        Russell Black made changes -
        Field Original Value New Value
        Attachment SOLR-3109.patch [ 12513763 ]
        Martijn van Groningen made changes -
        Assignee Martijn van Groningen [ martijn.v.groningen ]
        Martijn van Groningen made changes -
        Attachment SOLR-3109.patch [ 12513863 ]
        Russell Black made changes -
        Attachment SOLR-3109.patch [ 12513878 ]
        Martijn van Groningen made changes -
        Fix Version/s 3.6 [ 12319065 ]
        Fix Version/s 4.0 [ 12314992 ]
        Greg Bowyer made changes -
        Attachment SOLR-3109-Backport-of-grouping-performace-fix-to-3.x.patch [ 12514007 ]
        Attachment SOLR-3109-Backport-of-grouping-performace-fix-to-3.x.patch [ 12514008 ]
        Greg Bowyer made changes -
        Attachment SOLR-3109-Backport-of-grouping-performace-fix-to-3.x.patch [ 12514007 ]
        Russell Black made changes -
        Attachment SOLR-3109-lucene_solr_3_5.patch [ 12514023 ]
        Russell Black made changes -
        Attachment SOLR-3109.patch [ 12514112 ]
        Russell Black made changes -
        Attachment SOLR-3109.patch [ 12513878 ]
        Martijn van Groningen made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Martijn van Groningen
            Reporter:
            Russell Black
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development