Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8559

FCS facet performance optimization

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Implemented
    • 5.5, 6.0
    • 5.5
    • faceting

    Description

      While profiling a large collection (multi-sharded billions of documents), I found that a fast (5-10ms query) which had no matches would take 20-30 seconds when doing facets even when facet.mincount=1

      Profiling made it apparent that with facet.method=fcs 99% of the time was spent here.

      queue.udpateTop gets called numOfSegments*numTerms, the worst case when every term is in every segment. This formula doesn't take into account whether or not any of the terms have a positive count with respect to the docset.

      These optimizations are aimed to do two things:

      1. When mincount>0 don't include segments which all terms have zero counts. This should significantly speed up processing when terms are high cardinality and the matching docset is small
      2. FIXED TODO optimization: when mincount>0 move segment position the next non zero term value.

      both of these changes will minimize the number of called needed to the slow updateTop call.

      Attachments

        1. SOLR-8559-4-10-4.patch
          3 kB
          Keith Laban
        2. SOLR-8559.patch
          3 kB
          Dennis Gove
        3. solr-8559.patch
          3 kB
          Keith Laban

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dpgove Dennis Gove
            k317h Keith Laban
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment