I gave it a try on micro benchmarking, and it looks like we could gain a lot by switcing to sentinel approach for iterators, apart for being faster they are also a bit robuster to "one off" bugs.
This test is just a simulation made assuming docId is long (I have tried it with int and it is about the same result).
Just attaching it here as I did not want to create new issue for now, before we identify if there are some design/performance knock-out criteria.
test on my setup:
32bit java version "1.6.0_10-rc"
java(TM) SE Runtime Environment (build 1.6.0_10-rc-b28)
Windows XP Profesional 32bit
notebook, 3Gb RAM,
CPU x86 Family 6 Model 15 Stepping 11 GenuineIntel ~2194 Mhz
java -server -Xbatch
result (with docID long):
old total milliseconds=69203
new total milliseconds=53485
New/Old Time 53485/69203 (77.28711%)
all in all, faster more than 22% !!
Of course, this type of benchmark does not mean all iterator ops in real life are going to be 20% faster... other things probably dominate, but if it proves that this test does not have some flaws (easy possible)... well worth of pursuing