Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.2
    • Fix Version/s: 2.4.0
    • Component/s: ambari-metrics
    • Labels:
      None

      Description

      TopN is a useful feature in analyzing and finding the outlier in large clusters.

      2 Use Cases where meant to be solved through this story.
      Display the series for Top N hosts for a specific metric.
      Display the Top N metrics (Users / HBase table metrics) for a given host or aggregated across all hosts

      To facilitate the above, the following are supported.
      > Top "N" hosts when 1 metric is requested for "H" hosts.
      > Top "N" metrics when "M" metrics (or a pattern) are requested for 0 or 1 hosts.
      > Default TopN Hosts when More than a specific value of hosts are requested. Default = 20. Configurable through ams-site config (timeline.metrics.default.topn.hosts.limit)

      TopN API Example

      http://<AMS_HOST>:6188/ws/v1/timeline/metrics?metricNames=cpu_system._avg&hostname=h1,h2,h3,h4,h5&appId=HOST&startTime=1462879340&endTime=1462900940&topN=2&topNFunction=avg&isBottomN=false
      

      Added parameters
      topN - Mandatory parameter for TopN query which denotes the number of series to be returned.
      topNFunction - Optional parameter to specificy the comparator function for comparing 2 series while ordering. Default = max, Valid Values (max,avg,sum)
      isBottomN - Optional parameter. If bottom N series are required instead of top N. Default = false.

      1. AMBARI-16412-4.patch
        58 kB
        Aravindan Vijayan

        Issue Links

          Activity

          Hide
          avijayan Aravindan Vijayan added a comment -

          Pushed to trunk.

          Show
          avijayan Aravindan Vijayan added a comment - Pushed to trunk.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Ambari-trunk-Commit #4818 (See https://builds.apache.org/job/Ambari-trunk-Commit/4818/)
          AMBARI-16412 : Support TopN queries in AMS (avijayan) (avijayan: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=47c7b5ef6d17ca199a1b3b5799e7c66fe9ecc606)

          • ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TopNConfig.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/ConditionBuilder.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java
          • ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
          • ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
          • ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java
          • ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Ambari-trunk-Commit #4818 (See https://builds.apache.org/job/Ambari-trunk-Commit/4818/ ) AMBARI-16412 : Support TopN queries in AMS (avijayan) (avijayan: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=47c7b5ef6d17ca199a1b3b5799e7c66fe9ecc606 ) ambari-metrics/ambari-metrics-common/src/main/java/org/apache/hadoop/metrics2/sink/timeline/TopNConfig.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/ConditionBuilder.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/DefaultCondition.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStore.java ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestTimelineMetricStore.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TestPhoenixTransactSQL.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcherTest.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricStoreWatcher.java ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/webapp/TimelineWebServices.java

            People

            • Assignee:
              avijayan Aravindan Vijayan
              Reporter:
              avijayan Aravindan Vijayan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development