Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Redhat EL 5.1, Java 6

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Improved framework for data aggregation in Chuckwa.

      Description

      Chukwa is running data aggregation in database, and it has the ability to generate SQL statements base on macros. On the other hand, there is also Consolidator program which Chukwa uses to down sample data in the database. Consolidator program can be replaced with the generic Aggregator program plus some macros. This enhancement will make the database aggregation framework cleaner.

        Activity

        Hide
        Eric Yang added a comment -
        • Replace Consolidator with Aggregator macros
        • Updated database table schema
        • Fix interval for aggregation execute
        • Improved SQL macros support, add group_avg() and past_*_minutes key word
        Show
        Eric Yang added a comment - Replace Consolidator with Aggregator macros Updated database table schema Fix interval for aggregation execute Improved SQL macros support, add group_avg() and past_*_minutes key word
        Hide
        Eric Yang added a comment -
             [exec] -1 overall.  
        
             [exec]     +1 @author.  The patch does not contain any @author tags.
        
             [exec]     -1 tests included.  The patch doesn't appear to include any new or modified tests.
             [exec]                         Please justify why no tests are needed for this patch.
        
             [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
        
             [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
        
             [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
        
             [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
        
        Show
        Eric Yang added a comment - [exec] -1 overall. [exec] +1 @author. The patch does not contain any @author tags. [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no tests are needed for this patch. [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12395855/HADOOP-4827.patch
        against trunk revision 726075.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12395855/HADOOP-4827.patch against trunk revision 726075. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3726/console This message is automatically generated.
        Hide
        Jerome Boulon added a comment -

        +1 important patch for aggregation

        Show
        Jerome Boulon added a comment - +1 important patch for aggregation
        Hide
        Chris Douglas added a comment -

        I just committed this. Thanks, Eric

        Show
        Chris Douglas added a comment - I just committed this. Thanks, Eric
        Hide
        Robert Chansler added a comment -

        Edit release note for publication.

        What is new in HADOOP-4827:

        • Replace Consolidator with Aggregator macros
        • Updated database table schema
        • Fix interval for aggregation execute
        • Improved SQL macros support, add group_avg() and past_*_minutes key word
        Show
        Robert Chansler added a comment - Edit release note for publication. What is new in HADOOP-4827 : Replace Consolidator with Aggregator macros Updated database table schema Fix interval for aggregation execute Improved SQL macros support, add group_avg() and past_*_minutes key word

          People

          • Assignee:
            Eric Yang
            Reporter:
            Eric Yang
          • Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development