Lucene - Core
  1. Lucene - Core
  2. LUCENE-5303

OrdinalsCache should use reader.getCoreCacheKey()

        Details

        • Type: Bug Bug
        • Status: Resolved
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 4.6, 5.0
        • Component/s: modules/facet
        • Labels:
          None
        • Lucene Fields:
          New, Patch Available

          Description

          I'm doing some facet performance tests, and I tried using the CachedOrdsCountingFacetsAggregator to cache the decoded ords per doc X field ... but noticed it was generating way too many cache entries, because it's currently using the NDV instance as the cache key.

          NDV instances are thread-private so this results in way too many entries in the cache.

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Michael McCandless added a comment -

            Sorry, I meant BDV (BinaryDocValues) not NDV ...

            Show
            Michael McCandless added a comment - Sorry, I meant BDV (BinaryDocValues) not NDV ...
            Hide
            Permalink
            Shai Erera added a comment -

            Good catch!

            I guess we should use a compound cache key: coreCacheKey + clp.field?

            Show
            Shai Erera added a comment - Good catch! I guess we should use a compound cache key: coreCacheKey + clp.field?
            Hide
            Permalink
            Shai Erera added a comment -

            Patch changes the map to be a WeakHashMap<Object,Map<String,CachedOrds>> so the outer map is keyed by reader.getCoreCacheKey() and the inner map is from field to CachedOrds, where field is the BinaryDV field which holds the facets ordinals.

            I also added ramBytesUsed and a test which verifies that ramBytesUsed does not change between threads.

            Show
            Shai Erera added a comment - Patch changes the map to be a WeakHashMap<Object,Map<String,CachedOrds>> so the outer map is keyed by reader.getCoreCacheKey() and the inner map is from field to CachedOrds, where field is the BinaryDV field which holds the facets ordinals. I also added ramBytesUsed and a test which verifies that ramBytesUsed does not change between threads.
            Hide
            Permalink
            Michael McCandless added a comment -

            +1, thanks Shai!

            Show
            Michael McCandless added a comment - +1, thanks Shai!
            Hide
            Permalink
            ASF subversion and git services added a comment -

            Commit 1535163 from Shai Erera in branch 'dev/trunk'
            [ https://svn.apache.org/r1535163 ]

            LUCENE-5303: OrdinalsCache should use reader.getCoreCacheKey()

            Show
            ASF subversion and git services added a comment - Commit 1535163 from Shai Erera in branch 'dev/trunk' [ https://svn.apache.org/r1535163 ] LUCENE-5303 : OrdinalsCache should use reader.getCoreCacheKey()
            Hide
            Permalink
            ASF subversion and git services added a comment -

            Commit 1535167 from Shai Erera in branch 'dev/branches/branch_4x'
            [ https://svn.apache.org/r1535167 ]

            LUCENE-5303: OrdinalsCache should use reader.getCoreCacheKey()

            Show
            ASF subversion and git services added a comment - Commit 1535167 from Shai Erera in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1535167 ] LUCENE-5303 : OrdinalsCache should use reader.getCoreCacheKey()
            Hide
            Permalink
            Shai Erera added a comment -

            Committed to trunk and 4x.

            Show
            Shai Erera added a comment - Committed to trunk and 4x.
            Hide
            Permalink
            ASF subversion and git services added a comment -

            Commit 1535215 from Michael McCandless in branch 'dev/trunk'
            [ https://svn.apache.org/r1535215 ]

            LUCENE-5303: clear cache before each test

            Show
            ASF subversion and git services added a comment - Commit 1535215 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1535215 ] LUCENE-5303 : clear cache before each test
            Hide
            Permalink
            ASF subversion and git services added a comment -

            Commit 1535216 from Michael McCandless in branch 'dev/branches/branch_4x'
            [ https://svn.apache.org/r1535216 ]

            LUCENE-5303: clear cache before each test

            Show
            ASF subversion and git services added a comment - Commit 1535216 from Michael McCandless in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1535216 ] LUCENE-5303 : clear cache before each test

              People

              • Assignee:
                Shai Erera
                Reporter:
                Michael McCandless
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development