Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7730

speed-up faceting on doc values fields

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.2.1
    • Fix Version/s: 5.4
    • Component/s: faceting
    • Labels:

      Description

      every time we count facets on DocValues fields in Solr on many segments index we see the unnecessary hotspot:

      ....
              at org.apache.lucene.index.MultiFields.getMergedFieldInfos(MultiFields.java:248)
              at org.apache.lucene.index.SlowCompositeReaderWrapper.getFieldInfos(SlowCompositeReaderWrapper.java:239)
              at org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedSetDocValues(SlowCompositeReaderWrapper.java:176)
              at org.apache.solr.request.DocValuesFacets.getCounts(DocValuesFacets.java:72)
              at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:460) ....
      

      the reason is SlowCompositeReaderWrapper.getSortedSetDocValues() Line 136 and SlowCompositeReaderWrapper.getSortedDocValues() Line 174

      before return composite doc values, SCWR merges segment field infos, which is expensive, but after fieldinfo is merged, it checks only docvalue type in it. This dv type check can be done much easier in per segment basis.

      This patch gets some performance gain for those who count DV facets in Solr.

        Attachments

        1. LUCENE-7730.patch
          3 kB
          Yury Pakhomov
        2. LUCENE-7730.patch
          2 kB
          Mikhail Khludnev
        3. SOLR-7730.patch
          7 kB
          Mikhail Khludnev
        4. SOLR-7730-changes.patch
          1.0 kB
          Mikhail Khludnev

        Issue Links

          Activity

            People

            • Assignee:
              mkhl Mikhail Khludnev
              Reporter:
              mkhl Mikhail Khludnev

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment