Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-13890

Add postfilter support to {!terms} queries

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: master (9.0)
    • Fix Version/s: master (9.0), 8.5
    • Component/s: query parsers
    • Labels:
      None

      Description

      There are some use-cases where it'd be nice if the "terms" qparser created a query that could be run as a postfilter. Particularly, when users are checking for hundreds or thousands of terms, a postfilter implementation can be more performant than the standard processing.

      WIth this issue, I'd like to propose a post-filter implementation for the docValuesTermsFilter "method". Postfilter creation can use a SortedSetDocValues object to populate a DV bitset with the "terms" being checked for. Each document run through the post-filter can look at their doc-values for the field in question and check them efficiently against the constructed bitset.

        Attachments

        1. post_optimize_performance.png
          75 kB
          Jason Gerlowski
        2. Screen Shot 2020-01-02 at 2.25.12 PM.png
          62 kB
          Jason Gerlowski
        3. SOLR-13890.patch
          27 kB
          Jason Gerlowski
        4. SOLR-13890.patch
          38 kB
          Jason Gerlowski
        5. SOLR-13890.patch
          26 kB
          Jason Gerlowski
        6. SOLR-13890.patch
          19 kB
          Jason Gerlowski
        7. SOLR-13890.patch
          18 kB
          Jason Gerlowski
        8. SOLR-13890.patch
          13 kB
          Jason Gerlowski
        9. toplevel-tpi-perf-comparison.png
          38 kB
          Jason Gerlowski

          Issue Links

            Activity

              People

              • Assignee:
                gerlowskija Jason Gerlowski
                Reporter:
                gerlowskija Jason Gerlowski
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h
                  5h