Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13782

Make MutableRates metrics thread-local write, aggregate-on-read

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
    • Component/s: metrics
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently the MutableRates metrics class serializes all writes to metrics it contains because of its use of MetricsRegistry.add() (i.e., even two increments of unrelated metrics contained within the same MutableRates object will serialize w.r.t. each other). This class is used by RpcDetailedMetrics, which may have many hundreds of threads contending to modify these metrics. Instead we should allow updates to unrelated metrics objects to happen concurrently. To do so we can let each thread locally collect metrics, and on a snapshot, aggregate the metrics from all of the threads.

      I have collected some benchmark performance numbers in HADOOP-13747 (https://issues.apache.org/jira/secure/attachment/12835043/benchmark_results) which indicate that this can bring significantly higher performance in high contention situations.

        Attachments

        1. HADOOP-13782.006.patch
          16 kB
          Erik Krogen
        2. HADOOP-13782.005.patch
          15 kB
          Erik Krogen
        3. HADOOP-13782.004.patch
          15 kB
          Erik Krogen
        4. HADOOP-13782.003.patch
          16 kB
          Erik Krogen
        5. HADOOP-13782.002.patch
          15 kB
          Erik Krogen
        6. HADOOP-13782.001.patch
          17 kB
          Erik Krogen
        7. HADOOP-13782.000.patch
          17 kB
          Erik Krogen

          Issue Links

            Activity

              People

              • Assignee:
                xkrogen Erik Krogen
                Reporter:
                xkrogen Erik Krogen
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: