Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9592

decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.0, 6.1, 6.2
    • Fix Version/s: 6.3, 7.0
    • Component/s: Response Writers, search
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:

      Description

      I have serious performance issue using AtomicUpdate (and RealtimeGet) with non stored docValues.
      Because decorateDocValues try to merge each leafLeader on the fly via slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
      Simply access docValues via nonCompositeReader could resolve this issue.(patch)

      AtomicUpdate performance(or RealtimeGet performance)

      • Environment
        • solr version : 6.0.0
        • schema ~ 100 fields(90% docValues, some of those are multi valued)
        • index : 5,000,000
      • Performance
        • original : > 10sec per query
        • patched : at least 100msec per query

      This patch will also enhance search performance, because DocStreamer also fetch docValues via decorateDocValues.
      Though it depends on each environment, I could take 20% search performance gain.

      (This patch originally written for solr 6.0.0, and now rewritten for master)

        Attachments

        1. SOLR-9592_6x.patch
          34 kB
          Takahiro Ishikawa
        2. SOLR-9592.patch
          35 kB
          Takahiro Ishikawa
        3. SOLR-9592.patch
          3 kB
          Takahiro Ishikawa

          Activity

            People

            • Assignee:
              yseeley@gmail.com Yonik Seeley
              Reporter:
              takaishi Takahiro Ishikawa
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: