Hadoop Common
  1. Hadoop Common
  2. HADOOP-4843

Collect Job History log file and Job Conf file into Chukwa

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Redhat 5.1, Java 6

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Introduced Chuckwa collection of job history.

      Description

      Chukwa monitoring framework should collect job history log file, and job configuration files for analysis of the job performance and utilization.
      The current design is to tap into Job Tracker Instrumentation API. When a job is submitted, or completed, the JobTracker Instrumentation API handles the communication with Chukwa Agent to start or shutdown job log file streaming.

        Issue Links

          Activity

          Owen O'Malley made changes -
          Component/s contrib/chukwa [ 12312445 ]
          Nigel Daley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Robert Chansler made changes -
          Release Note Enable job history log file streaming into Chukwa by using JobTrackerInstrumentation API. Introduced Chuckwa collection of job history.
          Chris Douglas made changes -
          Fix Version/s 0.20.0 [ 12313438 ]
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Release Note What is new in HADOOP-4843:

          Enable job history log file streaming into Chukwa by using JobTrackerInstrumentation API.
          Enable job history log file streaming into Chukwa by using JobTrackerInstrumentation API.
          Resolution Fixed [ 1 ]
          Chris Douglas made changes -
          Hadoop Flags [Reviewed]
          Issue Type New Feature [ 2 ] Improvement [ 4 ]
          Eric Yang made changes -
          Release Note What is new in HADOOP-4843:

          Enable job history log file streaming into Chukwa by using JobTrackerInstrumentation API.
          Status Open [ 1 ] Patch Available [ 10002 ]
          Eric Yang made changes -
          Attachment HADOOP-4843.patch [ 12397930 ]
          Eric Yang made changes -
          Link This issue is blocked by HADOOP-4883 [ HADOOP-4883 ]
          Eric Yang made changes -
          Link This issue is blocked by HADOOP-4818 [ HADOOP-4818 ]
          Eric Yang made changes -
          Field Original Value New Value
          Summary Colect Job History log file and Job Conf file into Chukwa Collect Job History log file and Job Conf file into Chukwa
          Eric Yang created issue -

            People

            • Assignee:
              Eric Yang
              Reporter:
              Eric Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development