Lucene - Core
  1. Lucene - Core
  2. LUCENE-5303

OrdinalsCache should use reader.getCoreCacheKey()

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.6, Trunk
    • Component/s: modules/facet
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      I'm doing some facet performance tests, and I tried using the CachedOrdsCountingFacetsAggregator to cache the decoded ords per doc X field ... but noticed it was generating way too many cache entries, because it's currently using the NDV instance as the cache key.

      NDV instances are thread-private so this results in way too many entries in the cache.

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        1h 24m 1 Shai Erera 23/Oct/13 21:45
        Hide
        ASF subversion and git services added a comment -

        Commit 1535216 from Michael McCandless in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1535216 ]

        LUCENE-5303: clear cache before each test

        Show
        ASF subversion and git services added a comment - Commit 1535216 from Michael McCandless in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1535216 ] LUCENE-5303 : clear cache before each test
        Hide
        ASF subversion and git services added a comment -

        Commit 1535215 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1535215 ]

        LUCENE-5303: clear cache before each test

        Show
        ASF subversion and git services added a comment - Commit 1535215 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1535215 ] LUCENE-5303 : clear cache before each test
        Shai Erera made changes -
        Assignee Michael McCandless [ mikemccand ] Shai Erera [ shaie ]
        Resolution Fixed [ 1 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Lucene Fields New [ 10121 ] New,Patch Available [ 10121, 10120 ]
        Hide
        Shai Erera added a comment -

        Committed to trunk and 4x.

        Show
        Shai Erera added a comment - Committed to trunk and 4x.
        Hide
        ASF subversion and git services added a comment -

        Commit 1535167 from Shai Erera in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1535167 ]

        LUCENE-5303: OrdinalsCache should use reader.getCoreCacheKey()

        Show
        ASF subversion and git services added a comment - Commit 1535167 from Shai Erera in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1535167 ] LUCENE-5303 : OrdinalsCache should use reader.getCoreCacheKey()
        Hide
        ASF subversion and git services added a comment -

        Commit 1535163 from Shai Erera in branch 'dev/trunk'
        [ https://svn.apache.org/r1535163 ]

        LUCENE-5303: OrdinalsCache should use reader.getCoreCacheKey()

        Show
        ASF subversion and git services added a comment - Commit 1535163 from Shai Erera in branch 'dev/trunk' [ https://svn.apache.org/r1535163 ] LUCENE-5303 : OrdinalsCache should use reader.getCoreCacheKey()
        Hide
        Michael McCandless added a comment -

        +1, thanks Shai!

        Show
        Michael McCandless added a comment - +1, thanks Shai!
        Shai Erera made changes -
        Field Original Value New Value
        Attachment LUCENE-5303.patch [ 12609915 ]
        Hide
        Shai Erera added a comment -

        Patch changes the map to be a WeakHashMap<Object,Map<String,CachedOrds>> so the outer map is keyed by reader.getCoreCacheKey() and the inner map is from field to CachedOrds, where field is the BinaryDV field which holds the facets ordinals.

        I also added ramBytesUsed and a test which verifies that ramBytesUsed does not change between threads.

        Show
        Shai Erera added a comment - Patch changes the map to be a WeakHashMap<Object,Map<String,CachedOrds>> so the outer map is keyed by reader.getCoreCacheKey() and the inner map is from field to CachedOrds, where field is the BinaryDV field which holds the facets ordinals. I also added ramBytesUsed and a test which verifies that ramBytesUsed does not change between threads.
        Hide
        Shai Erera added a comment -

        Good catch!

        I guess we should use a compound cache key: coreCacheKey + clp.field?

        Show
        Shai Erera added a comment - Good catch! I guess we should use a compound cache key: coreCacheKey + clp.field?
        Hide
        Michael McCandless added a comment -

        Sorry, I meant BDV (BinaryDocValues) not NDV ...

        Show
        Michael McCandless added a comment - Sorry, I meant BDV (BinaryDocValues) not NDV ...
        Michael McCandless created issue -

          People

          • Assignee:
            Shai Erera
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development