Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-2593

disk full can cause index corruption in certain cases

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.9.4, 3.0.3, 3.1, 4.0-ALPHA
    • core/index
    • None
    • New

    Description

      Robert uncovered this nasty bug, in adding more randomness to
      oal.index tests...

      I got a standalone test to show the issue; the corruption path is
      as follows:

      • The merge hits an initial exception (eg disk full when merging the
        postings).
      • In handling this exception, IW closes all the sub-readers,
        suppressing any further exceptions.
      • If one of these sub-readers has pending deletions, which happens
        if readers are pooled in IW, it will flush them. If that flush
        hits a 2nd exception (eg disk full), then SegmentReader
        [incorrectly] leaves the SegmentInfo's delGen advanced by 1,
        referencing a corrupt file, yet the SegmentReader is still
        forcefully closed.
      • If enough disk frees up such that a later IW.commit/close
        succeeds, the resulting segments file will reference an invalid
        deletions file.

      Attachments

        1. LUCENE-2593.patch
          17 kB
          Michael McCandless

        Activity

          People

            mikemccand Michael McCandless
            mikemccand Michael McCandless
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment