Solr
  1. Solr
  2. SOLR-2089

Faceting: order term ords before converting to values

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None

      Description

      We should be able to speed up multi-valued faceting that sorts by count and returns many values by first sorting the term ords before converting them to a string.

      1. SOLR-2089.patch
        9 kB
        Yonik Seeley

        Activity

        Hide
        Yonik Seeley added a comment -

        Here's a draft patch that implements the sort before term val lookup.
        Currently untested for both speed and correctness.

        Show
        Yonik Seeley added a comment - Here's a draft patch that implements the sort before term val lookup. Currently untested for both speed and correctness.
        Hide
        Yonik Seeley added a comment -

        Results:
        docs=10M, docs matching query=1M, facet on field of 100,000 unique terms, facet.method=fc (multivalued)

        facet.limit ms to facet trunk ms to facet patch
        100 63 63
        1000 228 191
        5000 722 307
        10000 1033 316

        So a decent speedup when facet.limit is very high.
        It will also help when facet.limit is high relative to the number of unique terms (since the speedup is due to ordering the term ords and not having to seek as often).

        I plan on committing soon if there are no objections.

        Show
        Yonik Seeley added a comment - Results: docs=10M, docs matching query=1M, facet on field of 100,000 unique terms, facet.method=fc (multivalued) facet.limit ms to facet trunk ms to facet patch 100 63 63 1000 228 191 5000 722 307 10000 1033 316 So a decent speedup when facet.limit is very high. It will also help when facet.limit is high relative to the number of unique terms (since the speedup is due to ordering the term ords and not having to seek as often). I plan on committing soon if there are no objections.
        Hide
        Yonik Seeley added a comment -

        I've committed this to trunk. Will backport to 3x.

        Show
        Yonik Seeley added a comment - I've committed this to trunk. Will backport to 3x.
        Hide
        Yonik Seeley added a comment -

        OK, this has been backported to 3x, as well as fixing a bug that mark noticed (that was failing to trigger a test failure because of the way internal exceptions are handled in faceting). SolrTestCaseJ4.assertQ has been updated to fail if faceting is on and if an exception appears in the response.

        Show
        Yonik Seeley added a comment - OK, this has been backported to 3x, as well as fixing a bug that mark noticed (that was failing to trigger a test failure because of the way internal exceptions are handled in faceting). SolrTestCaseJ4.assertQ has been updated to fail if faceting is on and if an exception appears in the response.
        Hide
        Grant Ingersoll added a comment -

        Bulk close for 3.1.0 release

        Show
        Grant Ingersoll added a comment - Bulk close for 3.1.0 release

          People

          • Assignee:
            Unassigned
            Reporter:
            Yonik Seeley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development