[LUCENE-5654] CompoundFileWriter.close suppresses OOME

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.8.1, 4.9, 6.0
• Lucene Fields: New
• Assignee: Robert Muir
• Reporter: Michael McCandless
• Created: 07/May/14 21:19
• Updated: 28/Aug/22 14:06
• Resolved: 07/May/14 23:21

Description

I'm working on a test case, and it hit OOME while writing the compound
file; the test itself is separately buggy, but what was confounding
was that CompoundFileWriter continued trying to write to the
RAMOutputStream even after it had already hit OOME.

RAMOutputStream could be better here (e.g. only increment
currentBufferIndex if switchCurrentBuffer succeeds), but also I think
we should fix CompoundFileWriter.close to .closeWhileHandlingExc even
on non-IOExc.

It results in scary looking excs like this:

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 2428, Size: 2427
	at __randomizedtesting.SeedInfo.seed([4E313C43B4C1223B]:0)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:222)
Caused by: java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index: 2428, Size: 2427
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$CheckSegmentCount.run(TestIndexWriterThreadsToSegments.java:137)
	at java.util.concurrent.CyclicBarrier.dowait(CyclicBarrier.java:213)
	at java.util.concurrent.CyclicBarrier.await(CyclicBarrier.java:355)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$2.run(TestIndexWriterThreadsToSegments.java:219)
Caused by: java.lang.IndexOutOfBoundsException: Index: 2428, Size: 2427
	at java.util.ArrayList.rangeCheck(ArrayList.java:604)
	at java.util.ArrayList.get(ArrayList.java:382)
	at org.apache.lucene.store.RAMFile.getBuffer(RAMFile.java:68)
	at org.apache.lucene.store.RAMOutputStream.switchCurrentBuffer(RAMOutputStream.java:154)
	at org.apache.lucene.store.RAMOutputStream.writeBytes(RAMOutputStream.java:138)
	at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:125)
	at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:111)
	at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:70)
	at org.apache.lucene.codecs.CodecUtil.writeFooter(CodecUtil.java:184)
	at org.apache.lucene.store.CompoundFileWriter.close(CompoundFileWriter.java:144)
	at org.apache.lucene.store.CompoundFileDirectory.close(CompoundFileDirectory.java:178)
	at org.apache.lucene.util.IOUtils.closeWhileHandlingException(IOUtils.java:88)
	at org.apache.lucene.index.IndexWriter.createCompoundFile(IndexWriter.java:4512)
	at org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:485)
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:452)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:518)
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:629)
	at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:376)
	at org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:285)
	at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:260)
	at org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:250)
	at org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:124)
	at org.apache.lucene.index.TestIndexWriterThreadsToSegments$CheckSegmentCount.run(TestIndexWriterThreadsToSegments.java:124)