This is in 4.x now: just to try give some summary of this insanity...
- Uwe's original bug (
LUCENE-4455) caused mike to add a sizeInBytes test to TestRollingUpdates
- this test failed in jenkins overnight, because of some unreferenced files leftover from indexwriter. so it was clear MockDirectoryWrapper's unrefed files check has not really been working for some time! We now have a "test-the-tester" test for this so we know its working in the future!
- when we turned that on, all kinds of test failures happened: some exception/crash cases and what not, some real bugs like deleting non-index files. some fake test failures because tests made multiple commits. other fake test failures because IW couldnt delete files since MockDirectoryWrapper acts like windows (and some reader had them open).
- We fixed MockDirectoryWrapper more so that it handles the openDeletedFiles Windows case, and fixed it to handle indexes with multiple commits. More fake test failures because of hard-to-reproduce act-like-Windows corner cases like segments_N and segments.gen (its hard to reproduce the write at exactly the same time as these are slurped).
- To try to make those more reproducible, I figured MockDir could sometimes "hang onto" its IndexInputs a little bit longer so this is more likely to happen in tests, by very rarely doing a sleep in its close(). But a side effect of this, is testThreadInterruptDeadLock would sometimes interrupt this sleeping in close, causing an exception to be thrown in an IndexInput's close(). So this test (not intended for this purpose, but nice side effect) finds bugs where exception handling of close() for indexinputs is wrong, because of its unclosed files check.
And lots of pulling out hair and stuff in between.
Anyway it probably looks a lot worse than it is, because most of these failures are silly bugs/test bugs/trying to get the test framework right.
I have two jenkins pointed at branch_4x now:
Combined these do 70 test-core builds per hour. So every day is like a month's worth of test failures, which makes things look really unstable but its intentional
to try to hunt everything down at once instead of onesy-twosey for a long time
Lets give it a little time and see if it can flush anything out and then backport this stuff to 4.0, beast a bit more, and respin.