This field does not support sorting by distance, which is a very common use case.
We can add LatLonPoint.newDistanceSort(field, latitude, longitude) which returns a suitable SortField. There are a lot of optimizations esp when e.g. the priority queue gets full to avoid tons of haversin() computations.
Also, we can make use of the SortedNumeric data to switch newDistanceQuery/newPolygonQuery to the two-phase iterator api, so they aren't doing haversin() calls on bkd leaf nodes. It should look a lot like