Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-9405

IndexWriter incorrectly calls closeMergeReaders twice when the merged segment is 100% deleted

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: master (9.0), 8.6
    • Component/s: core/index
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      This is the first spinoff from a controversial PR to add a new index-time feature to Lucene to merge small segments during commit.  This can substantially reduce the number of small index segments to search.

      See specifically this discussion there.

      IndexWriter seems to be missing a success = true inside mergeMiddle in the case where all segments being merged have 100% deletions and the segments will simply be dropped.

      In this case, in master today, I think we are incorrectly calling closeMergedReaders twice, first with suppressExceptions = false and second time with true.

      There is a dedicated test case here showing the issue, but that test case relies on changes in the controversial feature (added MergePolicy.findFullFlushMerges). I think it should be possible to make another test case show the bug without that controversial feature, and I am unsure why our existing randomized tests have not uncovered this yet ...

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                simonw Simon Willnauer
                Reporter:
                mikemccand Michael McCandless
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m