After struggling for some time trying to fix the thread hazards
between flush & getAndLock, I decided to simply remove
I think it's too hairy/complex. The original motivation for this was
for when apps use a different thread to index different content
sources; this way we send similar docs to the same thread state and
should get better RAM efficiency.
But, 1) I think it's uncommon for apps to do this (they typically use
a "blind" thread pool) so it's fragile, 2) it's not clear really how
much perf gain this is (likely minor), and 3) it's too hard to get this working
correctly (this issue).
So instead I made DocumentsWriterPerThreadPool final, and I impl'd a
simple LIFO queue, and fixed the two places (in DW) that obtain a
thread state for indexing to use a new DWPTP.release API that puts the
state back into the free list & notifies for any waiting threads to
Note that other places that lock a thread state (e.g. flush/abort)
don't use this free list; I think this is OK?
Lucene tests pass, but this is a big change, and we need to scrutinize
/ test it more.
It seems to behave properly when I index Geonames w/ N threads, i.e. I
get N segments flushed by getReader, but I'll test more ... I'll also
make a standalone test case that validates this.
This is just an initial patch quickly coded up ... javadocs are wrong, etc.