In Flight benchmark, boost accumulator is used to estimate latency quantiles (0.5, 0.95, 0.99). Internally, boost adopts P-Square algorithm . P-Square is very bad at estimating skewed quantiles like 0.99, where TDigest shines.
Test result shows 0.99 latency is much better than what current code tells us. We should switch to TDigest.
- run flight-benchmark with default parameters
- calculate 0.99 quantile of latencies
- compare exact value (store all data points), value from tdigest, and value from boost
- test 5 rounds
TDigest gives more accurate values for all quantiles. For 0.5 quantiles, both TDigest and Boost gives very accurate result. For 0.95 quantiles, TDigest gives almost exact value, Boost has a bit deviation.