[#LUCENE-1198] Exception in DocumentsWriter.ThreadState.init leads to corruption

Lucene - Java

Exception in DocumentsWriter.ThreadState.init leads to corruption

Created: 01/Mar/08 01:30 PM Updated: 08/May/08 07:47 PM

Return to search

Component/s:

Index

Affects Version/s:

2.3

Fix Version/s:

2.3.2, 2.4

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	LUCENE-1198.patch	2008-03-01 01:32 PM	Michael McCandless	5 kB

Lucene Fields:	New
Resolution Date:	02/Mar/08 11:23 PM

Description

« Hide

If an exception is hit in the init method, DocumentsWriter incorrectly
increments numDocsInRAM when in fact the document is not added.

Spinoff of this thread:

http://markmail.org/message/e76hgkgldxhakuaa

The root cause that led to the exception in init was actually due to
incorrect use of Lucene's APIs (one thread still modifying the
Document while IndexWriter.addDocument is adding it) but still we
should protect against any exceptions coming out of init.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

Michael McCandless added a comment - 01/Mar/08 01:32 PM

Attached patch. I added a unit test showing the issue. To do this, I
added this method to IndexWriter:

boolean testPoint(String name)

Then in DocumentsWriter.ThreadState.init I added this:

assert writer.testPoint("DocumentsWriter.ThreadState.init start");

Then, tests can subclass IndexWriter and do interesting things at each
of these test points. We can add further test points over time...

Then I fixed the issue and the tests (& all tests) pass. I'll commit
in a day or two.

Michael McCandless made changes - 01/Mar/08 01:32 PM

Field	Original Value	New Value
Attachment		LUCENE-1198.patch [ 12376901 ]

Michael McCandless made changes - 02/Mar/08 11:23 PM

Resolution		Fixed [ 1 ]
Status	Open [ 1 ]	Resolved [ 5 ]

Repository	Revision	Date	User	Message
ASF	#632871	Sun Mar 02 23:24:07 UTC 2008	mikemccand	~~LUCENE-1198~~: don't increment numDocsInRAM unless DocumentsWriter.ThreadState.init succeeds
				Files Changed
				MODIFY /lucene/java/trunk/src/java/org/apache/lucene/index/DocumentsWriter.java MODIFY /lucene/java/trunk/src/java/org/apache/lucene/index/IndexWriter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexWriter.java MODIFY /lucene/java/trunk/src/test/org/apache/lucene/util/_TestUtil.java

Repository	Revision	Date	User	Message
ASF	#635869	Tue Mar 11 10:32:20 UTC 2008	mikemccand	~~LUCENE-1198~~ (porting to 2.3): don't corrupt the index if an exception happens inside DocumentsWriter.init
				Files Changed
				MODIFY /lucene/java/branches/lucene_2_3/src/java/org/apache/lucene/index/DocumentsWriter.java MODIFY /lucene/java/branches/lucene_2_3/src/test/org/apache/lucene/util/_TestUtil.java MODIFY /lucene/java/branches/lucene_2_3/src/java/org/apache/lucene/index/IndexWriter.java MODIFY /lucene/java/branches/lucene_2_3/src/test/org/apache/lucene/index/TestIndexWriter.java MODIFY /lucene/java/branches/lucene_2_3/CHANGES.txt

[ Permlink | « Hide ]

Hoss Man added a comment - 11/Mar/08 07:28 PM

targeted for 2.3.2 bug fix release

Hoss Man made changes - 11/Mar/08 07:28 PM

Fix Version/s

2.3.2 [ 12313057 ]

Michael Busch made changes - 08/May/08 07:47 PM

Status

Resolved [ 5 ]

Closed [ 6 ]