Hadoop Common
  1. Hadoop Common
  2. HADOOP-4843

Collect Job History log file and Job Conf file into Chukwa

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Redhat 5.1, Java 6

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Introduced Chuckwa collection of job history.

      Description

      Chukwa monitoring framework should collect job history log file, and job configuration files for analysis of the job performance and utilization.
      The current design is to tap into Job Tracker Instrumentation API. When a job is submitted, or completed, the JobTracker Instrumentation API handles the communication with Chukwa Agent to start or shutdown job log file streaming.

        Issue Links

          Activity

          Hide
          Eric Yang added a comment -

          For chukwa to collect Job History log file, the Job Instrumentation API needs to change to public.

          Show
          Eric Yang added a comment - For chukwa to collect Job History log file, the Job Instrumentation API needs to change to public.
          Hide
          Eric Yang added a comment -

          Implement JobTracker instrumentation API to communicate with Chukwa.

          Show
          Eric Yang added a comment - Implement JobTracker instrumentation API to communicate with Chukwa.
          Hide
          Ari Rabkin added a comment -

          Looks okay. +1

          Show
          Ari Rabkin added a comment - Looks okay. +1
          Hide
          Eric Yang added a comment -

          This patch depends on the patch for HADOOP-4818.

          Show
          Eric Yang added a comment - This patch depends on the patch for HADOOP-4818 .
          Hide
          Chris Douglas added a comment -

          I committed this. Thanks, Eric

          Show
          Chris Douglas added a comment - I committed this. Thanks, Eric

            People

            • Assignee:
              Eric Yang
              Reporter:
              Eric Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development