ARROW-9206 measures average latency in flight benchmark.
In practice, latency quantile is necessary to show the whole picture of rpc performance. E.g., 99% quantile, max, median.
A naive approach to save latencies of all batches is not applicable. Boost accumulator_set implements p square quantile algorithm which uses O(1) space with trivial computation overhead for each batch. It can be used in calculating latency quantiles.