Lucene - Core
  1. Lucene - Core
  2. LUCENE-1579

Cloned SegmentReaders fail to share FieldCache entries

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.9
    • Fix Version/s: 2.9
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      I just hit this on LUCENE-1516, which returns a cloned readOnly
      readers from IndexWriter.

      The problem is, when cloning, we create a new [thin] cloned
      SegmentReader for each segment. FieldCache keys directly off this
      object, so if you clone the reader and do a search that requires the
      FieldCache (eg, sorting) then that first search is always very slow
      because every single segment is reloading the FieldCache.

      This is of course a complete showstopper for LUCENE-1516.

      With LUCENE-831 we'll switch to a new FieldCache API; we should ensure
      this bug is not present there. We should also fix the bug in the
      current FieldCache API since for 2.9, users may hit this.

      1. LUCENE-1579.patch
        6 kB
        Michael McCandless
      2. LUCENE-1579.patch
        8 kB
        Michael McCandless
      3. LUCENE-1579.patch
        7 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Attached patch. I plan to commit in a day or two.

        I added a new deprecated expert public method to IndexReader:
        getFieldCacheWrapper(). Default impl is to return "this", but
        SegmentReader overrides that and returns a wrapper class that forwards
        hashCode()/equals() to the underlying freqStream.

        Show
        Michael McCandless added a comment - Attached patch. I plan to commit in a day or two. I added a new deprecated expert public method to IndexReader: getFieldCacheWrapper(). Default impl is to return "this", but SegmentReader overrides that and returns a wrapper class that forwards hashCode()/equals() to the underlying freqStream.
        Hide
        Michael McCandless added a comment -

        New patch. The last one was causing entries in FieldCache to get booted too soon.

        Show
        Michael McCandless added a comment - New patch. The last one was causing entries in FieldCache to get booted too soon.
        Hide
        Michael McCandless added a comment -

        New patch, even simpler.

        Show
        Michael McCandless added a comment - New patch, even simpler.

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development