Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-3109

group=true requests result in numerous redundant shard requests

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 3.5, 4.0-ALPHA
    • 3.6, 4.0-ALPHA
    • search
    • 64-bit Linux, sharded environment

    Description

      During the second phase of a group query, the collator sends a query to each of the shards. The purpose of this query is for shards to respond with the doc ids that match the set of group ids returned from the first phase. The problem is that it sends this second query to each shard multiple times. Specifically, in an environment with n shards, each shard will be hit with an identical query n times during the second phase of query processing, resulting in O(n 2) performance where n is the number of shards.

      I have traced this bug down to a single line in TopGroupsShardRequestFactory.java, and I am attaching a patch.

      Attachments

        1. SOLR-3109.patch
          1 kB
          Russell Black
        2. SOLR-3109.patch
          8 kB
          Martijn van Groningen
        3. SOLR-3109-Backport-of-grouping-performace-fix-to-3.x.patch
          12 kB
          Greg Bowyer
        4. SOLR-3109-lucene_solr_3_5.patch
          10 kB
          Russell Black
        5. SOLR-3109.patch
          11 kB
          Russell Black

        Activity

          People

            martijn.v.groningen Martijn van Groningen
            rblack Russell Black
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: