Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.2.0
    • Component/s: Data Processors
    • Labels:
      None

      Description

      Job conf contains some valuable values or can be used to infer new values like:

      • types of jobs (Pig,Streaming,Map only,singleton reducer)
      • mapred.output.key.class
      • mapred.map.runner.class
      • mapred.output.value.class
      • mapred.input.format.class
      • mapred.output.format.class
      • mapred.reducer.class
      • mapred.mapper.class

        Activity

        Hide
        zhangyongjiang Cheng added a comment -

        Submitted patch for review. New code added all kev/value pairs in the job conf to chukwa record.

        Show
        zhangyongjiang Cheng added a comment - Submitted patch for review. New code added all kev/value pairs in the job conf to chukwa record.
        Hide
        zhangyongjiang Cheng added a comment -

        Patch file submitted. Changes:

        • Mdl mapping in mdl.xml
        • Table template in database_create_tables.sql
        • Processor JobConfProcessor
        • Demux conf change in chukwa-demux-conf.xml
        • Aggregation sql in file aggregator.sql
        Show
        zhangyongjiang Cheng added a comment - Patch file submitted. Changes: Mdl mapping in mdl.xml Table template in database_create_tables.sql Processor JobConfProcessor Demux conf change in chukwa-demux-conf.xml Aggregation sql in file aggregator.sql
        Hide
        eyang Eric Yang added a comment -

        +1 looks good.

        Show
        eyang Eric Yang added a comment - +1 looks good.
        Hide
        eyang Eric Yang added a comment -

        I just committed this. Thanks Cheng.

        Show
        eyang Eric Yang added a comment - I just committed this. Thanks Cheng.
        Hide
        hudson Hudson added a comment -

        Integrated in Chukwa-trunk #24 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/24/)
        . Add new job conf metrics as database columns. (Cheng Zhang via Eric Yang)

        Show
        hudson Hudson added a comment - Integrated in Chukwa-trunk #24 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/24/ ) . Add new job conf metrics as database columns. (Cheng Zhang via Eric Yang)
        Hide
        hudson Hudson added a comment -
        Show
        hudson Hudson added a comment - Integrated in Chukwa-trunk #45 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/45/ )

          People

          • Assignee:
            zhangyongjiang Cheng
            Reporter:
            jboulon Jerome Boulon
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development