New patch attached, that merges in Mark's test & fix, and the original
spans test (converted to a unit test) from
LUCENE-1465, and adds
a method to IndexWriter to emulate the buggy behavior.
LUCENE-1255, this problem was just an "oddity" that
Lucene would record position -1 for the first token(s) if those tokens
all have position incrment 0. We started to fix it, realized it
breaks back-compat, and reverted it (accepting the "oddity").
Now, for this issue we are realizing the problem is much worse if a
payload happens to be attached to such tokens: instead of -1, the
position now comes back as Integer.MAX_VALUE (a side effect of how
payloads are stored in the index, which require that position delta be
non-zero), which then messes up *SpanQuery and I'm sure other things.
Subsequent tokens (once posIncr is > 0) then overflow int, and switch
I think this is a real and nasty bug, and we should fix it, despite
So in the patch, I've added deprecated
IndexWriter.setAllowMinus1Postion() to get back to the buggy
behaviour, if for some reason an application needs this, and then
fixed the bug by default.