Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-50

Create parser for parsing job history log file and job conf file in demux

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.1.2
    • Fix Version/s: None
    • Component/s: Data Processors
    • Labels:
      None
    • Environment:

      Redhat EL 5.1, Java 6

      Description

      The new job instrumentation API enable the ability to collect job history file and job conf files to Chukwa. The remaining work is to process the log files and job conf files in demux and produce key value pairs in sequence file for database to load the metrics data.

        Issue Links

          Activity

          Hide
          zhangyongjiang Cheng added a comment -

          Added parsers to parse job history and job conf data.

          Show
          zhangyongjiang Cheng added a comment - Added parsers to parse job history and job conf data.
          Hide
          eyang Eric Yang added a comment -

          +1 Looks good.

          Show
          eyang Eric Yang added a comment - +1 Looks good.
          Hide
          eyang Eric Yang added a comment -

          This patch contains both parsers for Job History and Job Conf files. It also added support for integer type columns to MetricsDataLoader. MDL Dictionary also mapped to create columns for both JobHistory log file for hadoop 0.18, and job history log file for hadoop 0.20.

          Show
          eyang Eric Yang added a comment - This patch contains both parsers for Job History and Job Conf files. It also added support for integer type columns to MetricsDataLoader. MDL Dictionary also mapped to create columns for both JobHistory log file for hadoop 0.18, and job history log file for hadoop 0.20.
          Hide
          eyang Eric Yang added a comment -

          Thanks Cheng, I just committed this.

          Show
          eyang Eric Yang added a comment - Thanks Cheng, I just committed this.

            People

            • Assignee:
              eyang Eric Yang
              Reporter:
              eyang Eric Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development