Hive
  1. Hive
  2. HIVE-2490

Add reset operation and average time attribute to Metrics MBean.

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      We should add a reset operation to the Metrics MBean, which will set all the counters to 0.

      Note: Deleting the counters from the map of attributes was not suggested because that could break any scripts that get the list of attributes from the bean and then the values of each attribute. Also, 0 is unlikely to be an actual value for any counter, and it will not break the increment functionality.

        Activity

        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2290/
        -----------------------------------------------------------

        Review request for hive and Paul Yang.

        Summary
        -------

        I added a reset operation which sets the value of all attributes to 0, see https://issues.apache.org/jira/browse/HIVE-2490 for an explanation of why 0 was chosen and why the attributes are not deleted.

        I also added an average time attribute in addition to number and total time. This seemed natural as we have all the data we need to calculate it.

        Previously, because we were incrementing the number at the beginning of the function, and the total time at the end of the function, it was impossible to guarantee an accurate average. In particular if the frequency of function calls was high and/or the execution time of the function was very high, the average that could be determined based on the given attributes could be very inaccurate.

        I also added some synchronizations to the Metrics class where I thought it did not appear to be thread safe.

        I based the average and reset API on what is done by Zookeeper in Hadoop.

        This addresses bug HIVE-2490.
        https://issues.apache.org/jira/browse/HIVE-2490

        Diffs


        trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 1179884
        trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java 1179884

        Diff: https://reviews.apache.org/r/2290/diff

        Testing
        -------

        I ran an instance of the Metastore, and executed queries against it from multiple clients, and verified the average attribute and reset operation behaved as expected.

        Thanks,

        Kevin

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2290/ ----------------------------------------------------------- Review request for hive and Paul Yang. Summary ------- I added a reset operation which sets the value of all attributes to 0, see https://issues.apache.org/jira/browse/HIVE-2490 for an explanation of why 0 was chosen and why the attributes are not deleted. I also added an average time attribute in addition to number and total time. This seemed natural as we have all the data we need to calculate it. Previously, because we were incrementing the number at the beginning of the function, and the total time at the end of the function, it was impossible to guarantee an accurate average. In particular if the frequency of function calls was high and/or the execution time of the function was very high, the average that could be determined based on the given attributes could be very inaccurate. I also added some synchronizations to the Metrics class where I thought it did not appear to be thread safe. I based the average and reset API on what is done by Zookeeper in Hadoop. This addresses bug HIVE-2490 . https://issues.apache.org/jira/browse/HIVE-2490 Diffs trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 1179884 trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java 1179884 Diff: https://reviews.apache.org/r/2290/diff Testing ------- I ran an instance of the Metastore, and executed queries against it from multiple clients, and verified the average attribute and reset operation behaved as expected. Thanks, Kevin
        Hide
        jiraposter@reviews.apache.org added a comment -

        -----------------------------------------------------------
        This is an automatically generated e-mail. To reply, visit:
        https://reviews.apache.org/r/2290/#review5298
        -----------------------------------------------------------

        Ship it!

        +1 Looks good. Can you upload the patch on jira so that I can test and commit.

        • Ashutosh

        On 2011-10-07 19:42:21, Kevin Wilfong wrote:

        -----------------------------------------------------------

        This is an automatically generated e-mail. To reply, visit:

        https://reviews.apache.org/r/2290/

        -----------------------------------------------------------

        (Updated 2011-10-07 19:42:21)

        Review request for hive and Paul Yang.

        Summary

        -------

        I added a reset operation which sets the value of all attributes to 0, see https://issues.apache.org/jira/browse/HIVE-2490 for an explanation of why 0 was chosen and why the attributes are not deleted.

        I also added an average time attribute in addition to number and total time. This seemed natural as we have all the data we need to calculate it.

        Previously, because we were incrementing the number at the beginning of the function, and the total time at the end of the function, it was impossible to guarantee an accurate average. In particular if the frequency of function calls was high and/or the execution time of the function was very high, the average that could be determined based on the given attributes could be very inaccurate.

        I also added some synchronizations to the Metrics class where I thought it did not appear to be thread safe.

        I based the average and reset API on what is done by Zookeeper in Hadoop.

        This addresses bug HIVE-2490.

        https://issues.apache.org/jira/browse/HIVE-2490

        Diffs

        -----

        trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 1179884

        trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java 1179884

        Diff: https://reviews.apache.org/r/2290/diff

        Testing

        -------

        I ran an instance of the Metastore, and executed queries against it from multiple clients, and verified the average attribute and reset operation behaved as expected.

        Thanks,

        Kevin

        Show
        jiraposter@reviews.apache.org added a comment - ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2290/#review5298 ----------------------------------------------------------- Ship it! +1 Looks good. Can you upload the patch on jira so that I can test and commit. Ashutosh On 2011-10-07 19:42:21, Kevin Wilfong wrote: ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2290/ ----------------------------------------------------------- (Updated 2011-10-07 19:42:21) Review request for hive and Paul Yang. Summary ------- I added a reset operation which sets the value of all attributes to 0, see https://issues.apache.org/jira/browse/HIVE-2490 for an explanation of why 0 was chosen and why the attributes are not deleted. I also added an average time attribute in addition to number and total time. This seemed natural as we have all the data we need to calculate it. Previously, because we were incrementing the number at the beginning of the function, and the total time at the end of the function, it was impossible to guarantee an accurate average. In particular if the frequency of function calls was high and/or the execution time of the function was very high, the average that could be determined based on the given attributes could be very inaccurate. I also added some synchronizations to the Metrics class where I thought it did not appear to be thread safe. I based the average and reset API on what is done by Zookeeper in Hadoop. This addresses bug HIVE-2490 . https://issues.apache.org/jira/browse/HIVE-2490 Diffs ----- trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 1179884 trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java 1179884 Diff: https://reviews.apache.org/r/2290/diff Testing ------- I ran an instance of the Metastore, and executed queries against it from multiple clients, and verified the average attribute and reset operation behaved as expected. Thanks, Kevin
        Hide
        Ashutosh Chauhan added a comment -

        Committed to trunk.

        Show
        Ashutosh Chauhan added a comment - Committed to trunk.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1274 (See https://builds.apache.org/job/Hive-trunk-h0.21/1274/)
        HIVE-2490: Add reset operation and average time attribute to Metrics MBean. (kevinwilfong via hashutosh) (Revision 1293352)

        Result = FAILURE
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293352
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java
        • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1274 (See https://builds.apache.org/job/Hive-trunk-h0.21/1274/ ) HIVE-2490 : Add reset operation and average time attribute to Metrics MBean. (kevinwilfong via hashutosh) (Revision 1293352) Result = FAILURE hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293352 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java
        Hide
        Ashutosh Chauhan added a comment -

        This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.

        Show
        Ashutosh Chauhan added a comment - This issue is closed now. It was released with the fix in 0.9.0. If there is a problem, please open a new jira and link this one with that.
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/)
        HIVE-2490: Add reset operation and average time attribute to Metrics MBean. (kevinwilfong via hashutosh) (Revision 1293352)

        Result = ABORTED
        hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293352
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java
        • /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #54 (See https://builds.apache.org/job/Hive-trunk-hadoop2/54/ ) HIVE-2490 : Add reset operation and average time attribute to Metrics MBean. (kevinwilfong via hashutosh) (Revision 1293352) Result = ABORTED hashutosh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293352 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java /hive/trunk/common/src/java/org/apache/hadoop/hive/common/metrics/MetricsMBeanImpl.java

          People

          • Assignee:
            Kevin Wilfong
            Reporter:
            Kevin Wilfong
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development