Lucene - Core
  1. Lucene - Core
  2. LUCENE-5303

OrdinalsCache should use reader.getCoreCacheKey()

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.6, Trunk
    • Component/s: modules/facet
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      I'm doing some facet performance tests, and I tried using the CachedOrdsCountingFacetsAggregator to cache the decoded ords per doc X field ... but noticed it was generating way too many cache entries, because it's currently using the NDV instance as the cache key.

      NDV instances are thread-private so this results in way too many entries in the cache.

        Activity

        Hide
        Michael McCandless added a comment -

        Sorry, I meant BDV (BinaryDocValues) not NDV ...

        Show
        Michael McCandless added a comment - Sorry, I meant BDV (BinaryDocValues) not NDV ...
        Hide
        Shai Erera added a comment -

        Good catch!

        I guess we should use a compound cache key: coreCacheKey + clp.field?

        Show
        Shai Erera added a comment - Good catch! I guess we should use a compound cache key: coreCacheKey + clp.field?
        Hide
        Shai Erera added a comment -

        Patch changes the map to be a WeakHashMap<Object,Map<String,CachedOrds>> so the outer map is keyed by reader.getCoreCacheKey() and the inner map is from field to CachedOrds, where field is the BinaryDV field which holds the facets ordinals.

        I also added ramBytesUsed and a test which verifies that ramBytesUsed does not change between threads.

        Show
        Shai Erera added a comment - Patch changes the map to be a WeakHashMap<Object,Map<String,CachedOrds>> so the outer map is keyed by reader.getCoreCacheKey() and the inner map is from field to CachedOrds, where field is the BinaryDV field which holds the facets ordinals. I also added ramBytesUsed and a test which verifies that ramBytesUsed does not change between threads.
        Hide
        Michael McCandless added a comment -

        +1, thanks Shai!

        Show
        Michael McCandless added a comment - +1, thanks Shai!
        Hide
        ASF subversion and git services added a comment -

        Commit 1535163 from Shai Erera in branch 'dev/trunk'
        [ https://svn.apache.org/r1535163 ]

        LUCENE-5303: OrdinalsCache should use reader.getCoreCacheKey()

        Show
        ASF subversion and git services added a comment - Commit 1535163 from Shai Erera in branch 'dev/trunk' [ https://svn.apache.org/r1535163 ] LUCENE-5303 : OrdinalsCache should use reader.getCoreCacheKey()
        Hide
        ASF subversion and git services added a comment -

        Commit 1535167 from Shai Erera in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1535167 ]

        LUCENE-5303: OrdinalsCache should use reader.getCoreCacheKey()

        Show
        ASF subversion and git services added a comment - Commit 1535167 from Shai Erera in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1535167 ] LUCENE-5303 : OrdinalsCache should use reader.getCoreCacheKey()
        Hide
        Shai Erera added a comment -

        Committed to trunk and 4x.

        Show
        Shai Erera added a comment - Committed to trunk and 4x.
        Hide
        ASF subversion and git services added a comment -

        Commit 1535215 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1535215 ]

        LUCENE-5303: clear cache before each test

        Show
        ASF subversion and git services added a comment - Commit 1535215 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1535215 ] LUCENE-5303 : clear cache before each test
        Hide
        ASF subversion and git services added a comment -

        Commit 1535216 from Michael McCandless in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1535216 ]

        LUCENE-5303: clear cache before each test

        Show
        ASF subversion and git services added a comment - Commit 1535216 from Michael McCandless in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1535216 ] LUCENE-5303 : clear cache before each test

          People

          • Assignee:
            Shai Erera
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development