Hadoop Common
  1. Hadoop Common
  2. HADOOP-3422

Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.17.2, 0.18.0, 0.18.1, 0.18.2
    • Fix Version/s: 0.20.0
    • Component/s: metrics
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Changed names of ganglia metrics to avoid conflicts and to better identify source function.

      Description

      The JobInProgress class reports all metrics with the name "value". The FileMetrics class puts all of the tags into the name when reporting the individual values, but the Ganglia Context does not put the tags into the name..

      This patch modifies the context to build names for the counter metrics out of the tag values. This enables the user to see the indivdual counter values with the ganglia web tool, on a per job basis

      1. ganglia-patch-3422-updated-v2.patch
        5 kB
        stack
      2. ganglia-patch-3422-updated.patch
        3 kB
        Brian Bockelman
      3. ganglia-patch-3422-qand-4137-1.patch
        2 kB
        Jason
      4. ganglia-patch-3422-and-4137.patch
        4 kB
        Brian Bockelman
      5. ganglia-patch-3422-and-4137.patch
        4 kB
        Brian Bockelman
      6. diff-20080520-1025.txt
        3 kB
        Jason

        Issue Links

          Activity

          Gavin made changes -
          Link This issue is depended upon by HADOOP-4675 [ HADOOP-4675 ]
          Gavin made changes -
          Link This issue blocks HADOOP-4675 [ HADOOP-4675 ]
          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Robert Chansler made changes -
          Hadoop Flags [Reviewed] [Incompatible change, Reviewed]
          Release Note Produce ganglia metrics with names of the form:
          jvm.metrics.*
          mapred.job.Job Counters .*
          mapred.job.Map-Reduce Framework.*
          mapred.job.Map-Reduce Framework.*
          mapred.job.Map-Reduce Framework.*
          mapred.jobtracker.*
          mapred.shuffleInput.*
          mapred.tasktracker.*

          instead of just using the last token name, or in the case of Per Job Counters, the string value, for all counters.

          The net result of this change is that the metrics names make it clear which part of hadoop is reporting, and the counter values each have their own RRD file instead of all the counters being aggregated under the name "value"
          Changed names of ganglia metrics to avoid conflicts and to better identify source function.
          stack made changes -
          Hadoop Flags [Reviewed]
          Fix Version/s 0.19.1 [ 12313473 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Fix Version/s hudson [ 12312940 ]
          Brian Bockelman made changes -
          Link This issue blocks HADOOP-4675 [ HADOOP-4675 ]
          stack made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          stack made changes -
          Assignee stack [ stack ]
          stack made changes -
          Status Patch Available [ 10002 ] In Progress [ 3 ]
          stack made changes -
          Attachment ganglia-patch-3422-updated-v2.patch [ 12394440 ]
          Brian Bockelman made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.18.2 [ 12313424 ]
          Affects Version/s 0.18.0 [ 12312972 ]
          Affects Version/s 0.17.2 [ 12313296 ]
          Fix Version/s 0.20.0 [ 12313438 ]
          Fix Version/s 0.19.1 [ 12313473 ]
          Fix Version/s hudson [ 12312940 ]
          Brian Bockelman made changes -
          Link This issue is duplicated by HADOOP-4137 [ HADOOP-4137 ]
          Brian Bockelman made changes -
          Attachment ganglia-patch-3422-updated.patch [ 12394071 ]
          Chris Douglas made changes -
          Link This issue is related to HADOOP-4137 [ HADOOP-4137 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jason made changes -
          Attachment ganglia-patch-3422-qand-4137-1.patch [ 12392450 ]
          Brian Bockelman made changes -
          Attachment ganglia-patch-3422-and-4137.patch [ 12391686 ]
          Brian Bockelman made changes -
          Affects Version/s 0.16.4 [ 12313132 ]
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.18.1 [ 12313357 ]
          Brian Bockelman made changes -
          Attachment ganglia-patch-3422-and-4137.patch [ 12391475 ]
          Mukund Madhugiri made changes -
          Fix Version/s 0.18.0 [ 12312972 ]
          Nigel Daley made changes -
          Fix Version/s 0.18.0 [ 12312972 ]
          Fix Version/s 0.17.0 [ 12312913 ]
          Jason made changes -
          Field Original Value New Value
          Attachment diff-20080520-1025.txt [ 12382394 ]
          Jason created issue -

            People

            • Assignee:
              stack
              Reporter:
              Jason
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development