I don't think that's the case (but I'm not near the code to double-check). LongFieldSource is only used in the ctor which takes a field name, and as documented - reads the values from the respective NDV field. Mike's test uses Expression and its ValueSource.
I think a solution could be to separate RangeAccumulator into a NumericRangeAccumulator and ValueSourceRangeAccumulator. The former always reads from an NDV and takes whatever range. The latter could restrict to only taking DoubleRange, and always call vs.doubleVal, since that seems like the only API a ValueSource has to impl. Or, it takes any Range, but always calls vs.doubleVal and converts it to Long, and the Range can convert it back?