Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-10241 Reduce performance/stability impact of latency metrics
  3. FLINK-10236

Reduce histogram percentile/quantile retrieval overhead

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.6.0, 1.7.0
    • None
    • Runtime / Metrics
    • None

    Description

      Most of our metrics reporters for histograms always report multiple quantiles: 0.5, 0.75, 0.90, 0.95, 0.98, 0.99, and 0.999.
      This is retrieved from HistogramStatistics/DescriptiveStatisticsHistogramStatistics but we do not have any optimisation for retrieving this many percentiles though and the plain use of DescriptiveStatistics#getPercentile has some constant overhead that could be avoided over multiple executions using Percentile#setData(double[]) to cache the current data set of the snapshot.

      In addition, min, max, mean, and standard deviation also each iterate over the array which could be done a single time.

      Attachments

        Issue Links

          is superceded by

          Sub-task - The sub-task of the issue FLINK-12982 Make DescriptiveStatisticsHistogramStatistics a true point-in-time snapshot

          • Major - Major loss of function.
          • Closed

          Activity

            People

              nkruber Nico Kruber
              nkruber Nico Kruber
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: