[#LUCENE-1208] Deadlock case in IndexWriter on exception just before flush

Lucene - Java

Deadlock case in IndexWriter on exception just before flush

Created: 07/Mar/08 07:08 PM Updated: 08/May/08 07:47 PM

Return to search

Component/s:

Index

Affects Version/s:

2.3, 2.3.1

Fix Version/s:

2.3.2, 2.4

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	LUCENE-1208.patch	2008-03-07 07:11 PM	Michael McCandless	2 kB

Lucene Fields:	New
Resolution Date:	07/Mar/08 09:46 PM

Description

« Hide

If a document hits a non-aborting exception, eg something goes wrong
in tokenStream.next(), and, that document had triggered a flush
(due to RAM or doc count) then DocumentsWriter will deadlock because
that thread marks the flush as pending but fails to clear it on
exception.

I have a simple test case showing this, and a fix fixing it.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

[ Permlink | « Hide ]

Michael McCandless added a comment - 07/Mar/08 07:11 PM

Attached patch. The fix is trivial. I will commit later today.

[ Permlink | « Hide ]

Michele Bini added a comment - 10/Mar/08 10:22 AM

As with ~~LUCENE-1210~~, shouldn't the patch be committed in the 2.3 branch, too, as it affects 2.3.1? Other issues, such as the speedups in ~~LUCENE-1211~~, although useful, can be left out as they are not bugs. But fix for deadlocks seem worthwhile for 2.3.x, too.

[ Permlink | « Hide ]

Michael McCandless added a comment - 10/Mar/08 10:51 AM

Agreed. I'm thinking these issues should be ported to 2.3.2:

~~LUCENE-1191~~
~~LUCENE-1197~~
~~LUCENE-1198~~
~~LUCENE-1199~~
~~LUCENE-1200~~
~~LUCENE-1208~~ (this issue)
~~LUCENE-1210~~

[ Permlink | « Hide ]

Michael Busch added a comment - 10/Mar/08 05:12 PM

We had seen this deadlock problem in our tests. I reran all tests with Lucene 2.3.1 + ~~LUCENE-1208~~ and didn't see the problem again so far!

[ Permlink | « Hide ]

Hoss Man added a comment - 11/Mar/08 07:29 PM

targeted for 2.3.2 bug fix release