[LUCENE-2593] disk full can cause index corruption in certain cases

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.4, 3.0.3, 3.1, 4.0-ALPHA
• Component/s: core/index
• Lucene Fields: New
• Assignee: Michael McCandless
• Reporter: Michael McCandless
• Created: 10/Aug/10 14:25
• Updated: 28/Aug/22 12:30
• Resolved: 10/Aug/10 22:34

Description

Robert uncovered this nasty bug, in adding more randomness to
oal.index tests...

I got a standalone test to show the issue; the corruption path is
as follows:

• The merge hits an initial exception (eg disk full when merging the
  postings).

• In handling this exception, IW closes all the sub-readers,
  suppressing any further exceptions.

• If one of these sub-readers has pending deletions, which happens
  if readers are pooled in IW, it will flush them. If that flush
  hits a 2nd exception (eg disk full), then SegmentReader
  [incorrectly] leaves the SegmentInfo's delGen advanced by 1,
  referencing a corrupt file, yet the SegmentReader is still
  forcefully closed.

• If enough disk frees up such that a later IW.commit/close
  succeeds, the resulting segments file will reference an invalid
  deletions file.