Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6728

Overhaul metrics framework

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.20.2, 0.21.0
    • None
    • None
    • None
    • Incompatible change
    • metrics

    Description

      Per discussions with Arun, Chris, Hong and Rajiv, et al, we concluded that the current metrics framework needs an overhaul to:

      • Allow multiple plugins for different monitoring systems simultaneously (see also: HADOOP-6508).
      • Refresh metrics plugin config without server restart.
        • Including filtering of metrics per plugin.
      • Support metrics schema for plugins.

      The jira will be resolved when core hadoop components (hdfs, mapreduce) are updated to use the new framework . Updates to external components that use the existing metrics framework will be tracked by different issues.

      The current design wiki.

      Attachments

        1. hadoop-6728-y20.104.patch
          649 kB
          Luke Lu
        2. metrics1-flow.png
          36 kB
          Luke Lu
        3. metrics2-builder.png
          31 kB
          Luke Lu
        4. metrics2-builder-r2.png
          39 kB
          Luke Lu
        5. metrics2-flow.png
          39 kB
          Luke Lu
        6. metrics2-mutable.png
          49 kB
          Luke Lu
        7. metrics2-mutable-r2.png
          58 kB
          Luke Lu
        8. metrics2-record.png
          50 kB
          Luke Lu
        9. metrics2-record-r2.png
          59 kB
          Luke Lu
        10. metrics2-uml.png
          129 kB
          Luke Lu
        11. metrics2-uml-r2.png
          154 kB
          Luke Lu

        Issue Links

          Activity

            People

              vicaya Luke Lu
              vicaya Luke Lu
              Votes:
              4 Vote for this issue
              Watchers:
              49 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: