Major Description
Today if you don't call score() some things are faster, we won't invoke similarity or read the norm for the document or other things.
On the other hand, its sad in this case that we are decompressing twice as many packed integers as we need (freqs can be skipped over, and our postings lists supports that) and walking all positions in phrase matching to determine the number of times the phrase matched (1 is enough, then we can stop).
When scoring is not needed, things can be optimized in other cases too (e.g. thats the whole concept of filters).