Lucene - Core
  1. Lucene - Core
  2. LUCENE-6290

Make the filter -> query migration less performance trappy

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.1, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The filter->query migration might be a bit trappy in terms of performance for our users.

      For instance a FilteredQuery over a TermQuery and a DocValuesRangeFilter should be migrated to a BooleanQuery with a MUST clause for the TermQuery and a FILTER clause for the DocValuesRangeQuery. Performance will be similar since in both case we would use the query to drive the iteration and the filter would only be used to check documents that match the query (we would NOT try to advance the filter iterator).

      However, if you only go half-way through the migration and end up with a FilteredQuery over a TermQuery and a QueryWrapperFilter(DocValuesRangeQuery) then performance will be terrible because this QueryWrapperFilter does not support random-access (which is the way such filters were not too slow before) and hides the approximation support from the DocValuesRangeQuery.

        Activity

        Hide
        Robert Muir added a comment -

        I think if QWF can somehow "pass thru" appoximation support of the wrapped query, that would be fantastic.

        Show
        Robert Muir added a comment - I think if QWF can somehow "pass thru" appoximation support of the wrapped query, that would be fantastic.
        Hide
        Adrien Grand added a comment -

        Here is a patch:

        • QueryWrapperFilter rewrites to a ConstantScoreQuery with a boost of 0 in order to propagate approximations
        • FilteredQuery rewrites to a BooleanQuery when the filter is a QueryWrapperFilter in order to take advantage of approximations
        Show
        Adrien Grand added a comment - Here is a patch: QueryWrapperFilter rewrites to a ConstantScoreQuery with a boost of 0 in order to propagate approximations FilteredQuery rewrites to a BooleanQuery when the filter is a QueryWrapperFilter in order to take advantage of approximations
        Hide
        Robert Muir added a comment -

        +1

        Show
        Robert Muir added a comment - +1
        Hide
        ASF subversion and git services added a comment -

        Commit 1662203 from Adrien Grand in branch 'dev/trunk'
        [ https://svn.apache.org/r1662203 ]

        LUCENE-6290: Make the filter -> query migration less trappy.

        Show
        ASF subversion and git services added a comment - Commit 1662203 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1662203 ] LUCENE-6290 : Make the filter -> query migration less trappy.
        Hide
        ASF subversion and git services added a comment -

        Commit 1662207 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1662207 ]

        LUCENE-6290: Make the filter -> query migration less trappy.

        Show
        ASF subversion and git services added a comment - Commit 1662207 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1662207 ] LUCENE-6290 : Make the filter -> query migration less trappy.
        Hide
        Timothy Potter added a comment -

        Bulk close after 5.1 release

        Show
        Timothy Potter added a comment - Bulk close after 5.1 release

          People

          • Assignee:
            Adrien Grand
            Reporter:
            Adrien Grand
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development