[LUCENE-5263] Deletes may be silently lost if an IOException is hit and later not hit (e.g., disk fills up and then frees up)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.5.1, 4.6, 6.0
• Component/s: core/index
• Lucene Fields: New
• Assignee: Michael McCandless
• Reporter: Michael McCandless
• Created: 07/Oct/13 23:04
• Updated: 28/Aug/22 13:54
• Resolved: 08/Oct/13 20:39

Description

This case is tricky to handle, yet I think realistic: disk fills up
temporarily, causes an exception in writeLiveDocs, and then the app
keeps using the IW instance.

Meanwhile disk later frees up again, IW is closed "successfully". In
certain cases, we can silently lose deletes in this case.

I had already committed
TestIndexWriterDeletes.testNoLostDeletesOnDiskFull, and Jenkins seems
happy with it so far, but when I added fangs to the test (cutover to
RandomIndexWriter from IndexWriter, allow IOE during getReader, add
randomness to when exc is thrown, etc.), it uncovered some real/nasty
bugs:

• ReaderPool.dropAll was suppressing any exception it hit, because
  

  if (priorE != null)

  should instead be

  if (priorE == null)

• After a merge, we have to write deletes before committing the
  segment, because an exception when writing deletes means we need
  to abort the merge

• Several places that were directly calling deleter.checkpoint must
  also increment the changeCount else on close IW thinks there are
  no changes and doesn't write a new segments file.

• closeInternal was dropping pooled readers after writing the
  segments file, which would lose deletes still buffered due to a
  previous exc.