Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-557

Make differentiation between recordName and contextName and clean up chukwa metrics

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.0
    • Component/s: Data Collection
    • Labels:
      None
    • Environment:

      Java 6, MacOSX 6

    • Release Note:
      Improved grouping of Hadoop metrics and Chukwa metrics.

      Description

      Metrics emited by Chukwa Agent through MetricsContext is structured as:

      contextName: chukwaHttpSender
      recordName: chukwaHttpSender

      The same context is repeated twice in the naming hierarchy.

      Ideally, the structure should be:

      contextName: chukwaAgent
      recordName: httpSender

      Therefore, the metrics can be collected in chukwa table, column family: agent_httpSender. In sequence file, it will be chukwa data type directory, and hashmap looks like:

      { "contextName":"chukwaAgent","recordName":"httpSender",...}
      

        Issue Links

          Activity

          Hide
          eyang Eric Yang added a comment -

          Chukwa Metrics also needs a cleaner parser.

          Show
          eyang Eric Yang added a comment - Chukwa Metrics also needs a cleaner parser.
          Hide
          eyang Eric Yang added a comment -

          This patch adds:

          ChukwaMetricsProcessor for demux.

          Chukwa metrics are defined as the following column family:

          • chukwaAgent_chunkQueue
          • chukwaAgent_httpSender
          • chukwaAgent_metrics

          Hadoop metrics are defined as the following column family:

          • jvm_metrics
          • mapred_metrics
          • dfs_metrics
          • dfs_namenode
          • dfs_FSDirectory
          • dfs_FSNamesystem
          • dfs_datanode
          • mapred_jobtracker
          • mapred_shuffleInput
          • mapred_shuffleOutput
          • mapred_tasktracker
          • mapred_job
          • rpc_metrics
          Show
          eyang Eric Yang added a comment - This patch adds: ChukwaMetricsProcessor for demux. Chukwa metrics are defined as the following column family: chukwaAgent_chunkQueue chukwaAgent_httpSender chukwaAgent_metrics Hadoop metrics are defined as the following column family: jvm_metrics mapred_metrics dfs_metrics dfs_namenode dfs_FSDirectory dfs_FSNamesystem dfs_datanode mapred_jobtracker mapred_shuffleInput mapred_shuffleOutput mapred_tasktracker mapred_job rpc_metrics
          Hide
          eyang Eric Yang added a comment -

          I just committed this.

          Show
          eyang Eric Yang added a comment - I just committed this.

            People

            • Assignee:
              eyang Eric Yang
              Reporter:
              eyang Eric Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development