Lucene - Core
  1. Lucene - Core
  2. LUCENE-6303

CachingWrapperFilter -> CachingWrapperQuery

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.1, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      As part of the filter -> query migration, we should migrate the caching wrappers (including the filter cache).

      I think the behaviour should be to delegate to the wrapped query when scores are needed and cache otherwise like CachingWrapperFilter does today.

      Also the cache should ignore query boosts so that field:value^2 and field:value^3 are considered equal if scores are not needed.

      1. LUCENE-6303.patch
        228 kB
        Adrien Grand
      2. LUCENE-6303-disable_auto-caching.patch
        12 kB
        Adrien Grand

        Activity

        Hide
        Robert Muir added a comment -

        +1. about the FilterCache:

        it would be nice, for example, if it worked out of box completely with simple defaults, with some way for users to configure/disable/plug in their own on indexsearcher (e.g. setCache or policy or whatever).

        this would give us freedom to cache 'implicit' filters, too using the logic we already have, for example if we see the same query over and over with needsScore=false just because it was in a booleanquery prohibited clause.

        Show
        Robert Muir added a comment - +1. about the FilterCache: it would be nice, for example, if it worked out of box completely with simple defaults, with some way for users to configure/disable/plug in their own on indexsearcher (e.g. setCache or policy or whatever). this would give us freedom to cache 'implicit' filters, too using the logic we already have, for example if we see the same query over and over with needsScore=false just because it was in a booleanquery prohibited clause.
        Hide
        Adrien Grand added a comment -

        Agreed on having filter caches pluggable on IndexSearcher. I'll give it a try.

        Show
        Adrien Grand added a comment - Agreed on having filter caches pluggable on IndexSearcher. I'll give it a try.
        Hide
        Adrien Grand added a comment -

        Here is a patch which:

        • replaces CachingWrapperFilter with CachingWrapperQuery
        • replaces FilterCache with QueryCache and caches weights instead of filters
        • removes DocIdSet.isCacheable since this method is not used anymore
        • adds built-in query caching to IndexSearcher (enabled by default): weights in the query tree that do not need scores are cached.
        Show
        Adrien Grand added a comment - Here is a patch which: replaces CachingWrapperFilter with CachingWrapperQuery replaces FilterCache with QueryCache and caches weights instead of filters removes DocIdSet.isCacheable since this method is not used anymore adds built-in query caching to IndexSearcher (enabled by default): weights in the query tree that do not need scores are cached.
        Hide
        Robert Muir added a comment -

        +1, this is awesome.

        Show
        Robert Muir added a comment - +1, this is awesome.
        Hide
        ASF subversion and git services added a comment -

        Commit 1662774 from Adrien Grand in branch 'dev/trunk'
        [ https://svn.apache.org/r1662774 ]

        LUCENE-6303: CachingWrapperFilter -> CachingWrapperQuery, FilterCache -> QueryCache and added caching to IndexSearcher.

        Show
        ASF subversion and git services added a comment - Commit 1662774 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1662774 ] LUCENE-6303 : CachingWrapperFilter -> CachingWrapperQuery, FilterCache -> QueryCache and added caching to IndexSearcher.
        Hide
        ASF subversion and git services added a comment -

        Commit 1662791 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1662791 ]

        LUCENE-6303: CachingWrapperFilter -> CachingWrapperQuery, FilterCache -> QueryCache and added caching to IndexSearcher.

        Show
        ASF subversion and git services added a comment - Commit 1662791 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1662791 ] LUCENE-6303 : CachingWrapperFilter -> CachingWrapperQuery, FilterCache -> QueryCache and added caching to IndexSearcher.
        Hide
        ASF subversion and git services added a comment -

        Commit 1662811 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1662811 ]

        LUCENE-6303: Fix compile errors on Java 7.

        Show
        ASF subversion and git services added a comment - Commit 1662811 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1662811 ] LUCENE-6303 : Fix compile errors on Java 7.
        Hide
        ASF subversion and git services added a comment -

        Commit 1662831 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1662831 ]

        LUCENE-6303: Fix 'access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.net.www.protocol.file")' error.

        Show
        ASF subversion and git services added a comment - Commit 1662831 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1662831 ] LUCENE-6303 : Fix 'access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.net.www.protocol.file")' error.
        Hide
        ASF subversion and git services added a comment -

        Commit 1662914 from Adrien Grand in branch 'dev/trunk'
        [ https://svn.apache.org/r1662914 ]

        LUCENE-6303: Do not cache filters twice in SolrIndexSearcher.

        Show
        ASF subversion and git services added a comment - Commit 1662914 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1662914 ] LUCENE-6303 : Do not cache filters twice in SolrIndexSearcher.
        Hide
        ASF subversion and git services added a comment -

        Commit 1662915 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1662915 ]

        LUCENE-6303: Do not cache filters twice in SolrIndexSearcher.

        Show
        ASF subversion and git services added a comment - Commit 1662915 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1662915 ] LUCENE-6303 : Do not cache filters twice in SolrIndexSearcher.
        Hide
        ASF subversion and git services added a comment -

        Commit 1663106 from Adrien Grand in branch 'dev/trunk'
        [ https://svn.apache.org/r1663106 ]

        LUCENE-6303: Make tests reproducible again.

        Show
        ASF subversion and git services added a comment - Commit 1663106 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1663106 ] LUCENE-6303 : Make tests reproducible again.
        Hide
        ASF subversion and git services added a comment -

        Commit 1663122 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1663122 ]

        LUCENE-6303: Make tests reproducible again.

        Show
        ASF subversion and git services added a comment - Commit 1663122 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1663122 ] LUCENE-6303 : Make tests reproducible again.
        Hide
        Adrien Grand added a comment -

        Reopen: This change triggered issues with tests that modify queries in-place after they have been put into the cache. There have been some attempts to fix it (see LUCENE-6369) but there doesn't seem to be any quick path to a fix so we should disable automatic caching for now.

        Show
        Adrien Grand added a comment - Reopen: This change triggered issues with tests that modify queries in-place after they have been put into the cache. There have been some attempts to fix it (see LUCENE-6369 ) but there doesn't seem to be any quick path to a fix so we should disable automatic caching for now.
        Hide
        Adrien Grand added a comment -

        Here is a patch which disables auto caching and tries to better detect mutated queries after they have been put into the cache.

        Show
        Adrien Grand added a comment - Here is a patch which disables auto caching and tries to better detect mutated queries after they have been put into the cache.
        Hide
        ASF subversion and git services added a comment -

        Commit 1670006 from Adrien Grand in branch 'dev/trunk'
        [ https://svn.apache.org/r1670006 ]

        LUCENE-6303: Disable auto-caching in IndexSearcher.

        Show
        ASF subversion and git services added a comment - Commit 1670006 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1670006 ] LUCENE-6303 : Disable auto-caching in IndexSearcher.
        Hide
        ASF subversion and git services added a comment -

        Commit 1670007 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1670007 ]

        LUCENE-6303: Disable auto-caching in IndexSearcher.

        Show
        ASF subversion and git services added a comment - Commit 1670007 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1670007 ] LUCENE-6303 : Disable auto-caching in IndexSearcher.
        Hide
        ASF subversion and git services added a comment -

        Commit 1670009 from Adrien Grand in branch 'dev/trunk'
        [ https://svn.apache.org/r1670009 ]

        LUCENE-6303: Fix changelog.

        Show
        ASF subversion and git services added a comment - Commit 1670009 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1670009 ] LUCENE-6303 : Fix changelog.
        Hide
        ASF subversion and git services added a comment -

        Commit 1670010 from Adrien Grand in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1670010 ]

        LUCENE-6303: Fix changelog.

        Show
        ASF subversion and git services added a comment - Commit 1670010 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1670010 ] LUCENE-6303 : Fix changelog.
        Hide
        Adrien Grand added a comment -

        Committed: auto caching is now disabled until we figure out how we can fix queries so that they are good to use as cache keys.

        Show
        Adrien Grand added a comment - Committed: auto caching is now disabled until we figure out how we can fix queries so that they are good to use as cache keys.
        Hide
        Timothy Potter added a comment -

        Bulk close after 5.1 release

        Show
        Timothy Potter added a comment - Bulk close after 5.1 release

          People

          • Assignee:
            Adrien Grand
            Reporter:
            Adrien Grand
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development