Lucene - Core
  1. Lucene - Core
  2. LUCENE-1700

LogMergePolicy.findMergesToExpungeDeletes need to get deletes from the SegmentReader

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.4.1
    • Fix Version/s: 2.9
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      With LUCENE-1516, deletes are carried over in the SegmentReaders
      which means implementations of
      MergePolicy.findMergesToExpungeDeletes (such as LogMergePolicy)
      need to obtain deletion info from the SR (instead of from the
      SegmentInfo which won't have the information).

      1. LUCENE-1700.patch
        3 kB
        Michael McCandless

        Activity

        Hide
        Jason Rutherglen added a comment -

        Taking a step back, maybe we can solve the package protected
        SegmentInfo issue here by creating a new class with the
        necessary attributes?

        Here's what LUCENE-1313 does:

         SegmentReader sr = writer.readerPool.getIfExists(info);
        if (info.hasDeletions() || (sr != null && sr.hasDeletions())) {
        

        Because SegmentInfo is package protected it seems ok to access a
        package protected method (or in this case variable) in
        IndexWriter.

        Show
        Jason Rutherglen added a comment - Taking a step back, maybe we can solve the package protected SegmentInfo issue here by creating a new class with the necessary attributes? Here's what LUCENE-1313 does: SegmentReader sr = writer.readerPool.getIfExists(info); if (info.hasDeletions() || (sr != null && sr.hasDeletions())) { Because SegmentInfo is package protected it seems ok to access a package protected method (or in this case variable) in IndexWriter.
        Hide
        Michael McCandless added a comment -

        Attached patch.

        I added a test case showing it, then took that same approach (from LUCENE-1313) and the test passes.

        I also found that with NRT, because the deletions are applied before
        building the CFS after flushing, we wind up holding open both the
        non-CFS and CFS files on creating the reader. So, I changed deletions
        to flush after the CFS is built.

        I plan to commit in a day or two.

        Show
        Michael McCandless added a comment - Attached patch. I added a test case showing it, then took that same approach (from LUCENE-1313 ) and the test passes. I also found that with NRT, because the deletions are applied before building the CFS after flushing, we wind up holding open both the non-CFS and CFS files on creating the reader. So, I changed deletions to flush after the CFS is built. I plan to commit in a day or two.
        Hide
        Michael McCandless added a comment -

        Thanks Jason!

        Show
        Michael McCandless added a comment - Thanks Jason!

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Jason Rutherglen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 48h
              48h
              Remaining:
              Remaining Estimate - 48h
              48h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development