The Lucene tests do not take care to close all file handles. Unless I am missing something, every single instance of Directory, IndexReader, IndexWriter, IndexSearcher, TermPositions, etc. should be wrapped with a try-finally pattern, such that the instance is always closed. Not doing so risks leaving files open, depending on the GC behavior. I believe this causes tests to fail with a "could not delete" exception, inconsistently. I at least observe this on a fast machine with Windows, where deletion is a little more sensitive to open handles. It seems dangerous and undesirable, anyway (again, unless I am missing something). I don't know of another pattern in Java that would actually be safe.
Some of these objects may just happen to be safe to let dangle in the wind, until the GC reaps, but by the contracts that really can't be allowed. The close methods need to be called to release resources.
Fixing this appears to alleviate the test failures, but it is hard to tell due to the nondeterministic behavior. I am reluctant to make up the whole patch if this is inaccurate - it is somewhat tedious. The classes involved can be instrumented to expose this problem. (In particular, I would imagine that the finalizer should never be reached without the close() methods being previously invoked.)