Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13747

Use LongAdder for more efficient metrics tracking

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • metrics
    • None

    Description

      Currently many metrics, including RpcMetrics and RpcDetailedMetrics, use a synchronized counter to be updated by all handler threads (multiple hundreds in large production clusters). As andrew.wang suggested, it'd be more efficient to use the LongAdder library which dynamically create intermediate-result variables.

      Assigning to xkrogen who has already done some investigation on this.

      Attachments

        1. HADOOP-13747.patch
          7 kB
          Erik Krogen
        2. benchmark_results
          6 kB
          Erik Krogen

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13615 Convert uses of AtomicLong for counter metrics to LongAdder

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13782 Make MutableRates metrics thread-local write, aggregate-on-read

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              xkrogen Erik Krogen
              zhz Zhe Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: