Actually I consider this a bug in IW's ACID transactional semantics...
The "I" in ACID is "isolation", meaning, a reader shouldn't ever see
uncommitted changes from a writer.
This bug breaks "isolation", ie, if you open with CREATE, today, IW
may or may not slip in a commit on you, depending (rather
unexpectedly, and likely the opposite of what you'd guess) on whether
a prior index is already there.
So IW sometimes will break ACID transactions, and sometimes not, if
you use CREATE.
Further, IW really shouldn't ever write a commit "automatically" – it
used to do this (with autoCommit=true), but we stopped doing so
(except for this bug), now that autoCommit is false.
With this fix, IW will never sneak in a commit. Your app fully
controls when the transaction (including CREATE, which in Lucene,
unlike most RDMBSs, is part of the transaction) becomes visible.
Thats not a very strong argument for a back compat break on a minor release though...
Hmmm... I think the back compat break is very minor. Also, IW's
now-gone autoCommit never "promised" when commits would be made, so
this was really undocumented behavior.
Solr's test failures were not due to this break; they we actually due
to a sneaky bug that otherwise (had we not had Solr's tests) would
likely have remained undiscovered for quite some time.
And, much of the "noise" in the patch is from tests relying on exact
file names, commit counts, etc. Plus some of the usual Shai-cleanups
I guess if we really care to, we can emulate the "not quite ACID" bug
when Version <= 3.1?
The question is also, what happens if you call IndexWriter.getReader() without the initial commit? Does this work with your patch?
This should be perfectly fine – you'll get a reader searching 0 docs.
Shai can you add a test case confirming this?
Shai a couple other things:
- Please shrink-wrap the try/except in IFD
- In Directory.listCommits, instead of catching
IndexNotFoundException and returning empty list, I think we should
throw it? You get this exception today, right?