Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 7.0
    • Fix Version/s: 7.0
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Somehow related to LUCENE-7407, after the transition, the uninvert time of numeric fields has gone up substantially. I haven't tested all field types yet, just integer fields, which show a 55% performance regression for the initial uninvert time.

      This was tested with a numeric field of cardinality 1M on a 10M doc index.

      q=id:1&sort=my_numeric_field desc
      

        Issue Links

          Activity

          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          OK, I found the culprit...
          https://github.com/apache/lucene-solr/commit/f7aa200d406dbd05a35d6116198302d90b92cb29#diff-595e0e789c5e7ac91fe0300782f1bea6R640

          This causes the field to be traversed twice... the first time for docsWithValue, and the second time for the actual uninversion (which also calculates docsWithValue anyway but then doesn't use it).

          Show
          yseeley@gmail.com Yonik Seeley added a comment - OK, I found the culprit... https://github.com/apache/lucene-solr/commit/f7aa200d406dbd05a35d6116198302d90b92cb29#diff-595e0e789c5e7ac91fe0300782f1bea6R640 This causes the field to be traversed twice... the first time for docsWithValue, and the second time for the actual uninversion (which also calculates docsWithValue anyway but then doesn't use it).
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1d2e440a8fe3df8d3207a7428841f79f63381e4f in lucene-solr's branch refs/heads/master from Yonik Seeley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1d2e440 ]

          SOLR-9837: fix redundant calculation of docsWithField for numeric fields in field cache

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1d2e440a8fe3df8d3207a7428841f79f63381e4f in lucene-solr's branch refs/heads/master from Yonik Seeley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1d2e440 ] SOLR-9837 : fix redundant calculation of docsWithField for numeric fields in field cache
          Hide
          elyograg Shawn Heisey added a comment - - edited

          What would it take to do something like the following for Solr?

          https://home.apache.org/~mikemccand/lucenebench/

          Show
          elyograg Shawn Heisey added a comment - - edited What would it take to do something like the following for Solr? https://home.apache.org/~mikemccand/lucenebench/
          Hide
          tomasflobbe Tomás Fernández Löbbe added a comment -

          That would be great. I always wanted something like that for Solr too, but I never had the time to work on it.

          Show
          tomasflobbe Tomás Fernández Löbbe added a comment - That would be great. I always wanted something like that for Solr too, but I never had the time to work on it.

            People

            • Assignee:
              yseeley@gmail.com Yonik Seeley
              Reporter:
              yseeley@gmail.com Yonik Seeley
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development