Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6290

Make the filter -> query migration less performance trappy

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.1, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The filter->query migration might be a bit trappy in terms of performance for our users.

      For instance a FilteredQuery over a TermQuery and a DocValuesRangeFilter should be migrated to a BooleanQuery with a MUST clause for the TermQuery and a FILTER clause for the DocValuesRangeQuery. Performance will be similar since in both case we would use the query to drive the iteration and the filter would only be used to check documents that match the query (we would NOT try to advance the filter iterator).

      However, if you only go half-way through the migration and end up with a FilteredQuery over a TermQuery and a QueryWrapperFilter(DocValuesRangeQuery) then performance will be terrible because this QueryWrapperFilter does not support random-access (which is the way such filters were not too slow before) and hides the approximation support from the DocValuesRangeQuery.

        Activity

        Hide
        rcmuir Robert Muir added a comment -

        I think if QWF can somehow "pass thru" appoximation support of the wrapped query, that would be fantastic.

        Show
        rcmuir Robert Muir added a comment - I think if QWF can somehow "pass thru" appoximation support of the wrapped query, that would be fantastic.
        Hide
        jpountz Adrien Grand added a comment -

        Here is a patch:

        • QueryWrapperFilter rewrites to a ConstantScoreQuery with a boost of 0 in order to propagate approximations
        • FilteredQuery rewrites to a BooleanQuery when the filter is a QueryWrapperFilter in order to take advantage of approximations
        Show
        jpountz Adrien Grand added a comment - Here is a patch: QueryWrapperFilter rewrites to a ConstantScoreQuery with a boost of 0 in order to propagate approximations FilteredQuery rewrites to a BooleanQuery when the filter is a QueryWrapperFilter in order to take advantage of approximations
        Hide
        rcmuir Robert Muir added a comment -

        +1

        Show
        rcmuir Robert Muir added a comment - +1
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1662203 from Adrien Grand in branch 'dev/trunk'
        [ https://svn.apache.org/r1662203 ]

        LUCENE-6290: Make the filter -> query migration less trappy.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1662203 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1662203 ] LUCENE-6290 : Make the filter -> query migration less trappy.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1662207 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1662207 ]

        LUCENE-6290: Make the filter -> query migration less trappy.

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1662207 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1662207 ] LUCENE-6290 : Make the filter -> query migration less trappy.
        Hide
        thelabdude Timothy Potter added a comment -

        Bulk close after 5.1 release

        Show
        thelabdude Timothy Potter added a comment - Bulk close after 5.1 release

          People

          • Assignee:
            jpountz Adrien Grand
            Reporter:
            jpountz Adrien Grand
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development