I found this while exploring
LUCENE-5748, but it currently applies to both variable length BINARY and SORTED_SET, so I think its worth it to do here first.
I think its just a holdover from before MonotonicBlockPackedWriter that to access element N we currently do:
Thats because previously we didnt have packed ints that supported > Integer.MAX_VALUE elements. But thats been fixed for a long time. If we just write a 0 first and do this:
The access is then much faster. For sorting i see around 20% improvement. We don't lose any compression because we should assume the delta from 0 .. 1 is similar to any other gap N .. N+1