Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-17382

Migrate AMS queries to use ROW_TIMESTAMP instead of native timerange hint

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.4.0
    • metrics-3.0.0
    • None

    Description

      With PHOENIX-914, there is a change in implementation , As earlier, timestamp range was passed as a hint to the query to get advantage of native timerange optimization in hbase but with new implementation we can mark the timestamp column in the schema as a ROW_TIMESTAMP and pass timestamp range with “where” clause only to achieve equivalent performance and better accuracy.

      For eq:-
      With earlier implementation , AMS forms query like this:-

      SELECT /*+ NATIVE_TIME_RANGE(1448029523000) */ METRIC_NAME, APP_ID, INSTANCE_ID, SERVER_TIME, UNITS, METRIC_SUM, HOSTS_COUNT, METRIC_MAX, METRIC_MIN FROM METRIC_AGGREGATE WHERE (METRIC_NAME IN ('regionserver.Server.totalRequestCount', 'regionserver.Server.blockCacheCountHitPercent', 'regionserver.Server.regionCount', 'regionserver.Server.compactionQueueLength', 'regionserver.Server.storeFileCount', 'master.Server.averageLoad')) AND APP_ID = 'ams-hbase' AND SERVER_TIME >= 1448029643000 AND SERVER_TIME < 1448033243 ORDER BY METRIC_NAME, SERVER_TIME LIMIT 11520

      But with PHOENIX-914 :-

      Declare SERVER_TIME as ROW_TIMESTAMP in schema:-
      CREATE TABLE DESTINATION_METRICS_TABLE (SERVER_TIME DATE not null, METRIC_ID CHAR(15) not null, METRIC_VALUE bigint … CONSTRAINT PK PRIMARY KEY(CREATED_DATE ROW_TIMESTAMP, METRIC_ID…)) …;

      And remove hint from the query:-
      SELECT METRIC_NAME, APP_ID, INSTANCE_ID, SERVER_TIME, UNITS, METRIC_SUM, HOSTS_COUNT, METRIC_MAX, METRIC_MIN FROM METRIC_AGGREGATE WHERE (METRIC_NAME IN ('regionserver.Server.totalRequestCount', 'regionserver.Server.blockCacheCountHitPercent', 'regionserver.Server.regionCount', 'regionserver.Server.compactionQueueLength', 'regionserver.Server.storeFileCount', 'master.Server.averageLoad')) AND APP_ID = 'ams-hbase' AND SERVER_TIME >= 1448029643000 AND SERVER_TIME < 1448033243 ORDER BY METRIC_NAME, SERVER_TIME LIMIT 11520

      Attachments

        1. AMBARI-17382-1.patch
          57 kB
          Aravindan Vijayan

        Issue Links

          Activity

            People

              avijayan Aravindan Vijayan
              avijayan Aravindan Vijayan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: