Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4602

Use DocValues to store per-doc facet ord

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.2, 6.0
    • Component/s: modules/facet
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      Spinoff from LUCENE-4600

      DocValues can be used to hold the byte[] encoding all facet ords for
      the document, instead of payloads. I made a hacked up approximation
      of in-RAM DV (see CachedCountingFacetsCollector in the patch) and the
      gains were somewhat surprisingly large:

                          Task    QPS base      StdDev    QPS comp      StdDev                Pct diff
                      HighTerm        0.53      (0.9%)        1.00      (2.5%)   87.3% (  83% -   91%)
                       LowTerm        7.59      (0.6%)       26.75     (12.9%)  252.6% ( 237% -  267%)
                       MedTerm        3.35      (0.7%)       12.71      (9.0%)  279.8% ( 268% -  291%)
      

      I didn't think payloads were THAT slow; I think it must be the advance
      implementation?

      We need to separately test on-disk DV to make sure it's at least
      on-par with payloads (but hopefully faster) and if so ... we should
      cutover facets to using DV.

        Attachments

        1. FacetsPayloadMigrationReader.java
          7 kB
          Shai Erera
        2. LUCENE-4602.patch
          159 kB
          Shai Erera
        3. LUCENE-4602.patch
          102 kB
          Shai Erera
        4. LUCENE-4602.patch
          99 kB
          Shai Erera
        5. LUCENE-4602.patch
          20 kB
          Michael McCandless
        6. LUCENE-4602.patch
          24 kB
          Michael McCandless
        7. TestFacetsPayloadMigrationReader.java
          17 kB
          Shai Erera

          Issue Links

            Activity

              People

              • Assignee:
                shaie Shai Erera
                Reporter:
                mikemccand Michael McCandless
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: