Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-25611

After purging AMS database "TimelineMetricMetadataKey is null" error is thrown

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Workaround
    • Affects Version/s: 2.7.5
    • Fix Version/s: None
    • Component/s: ambari-metrics
    • Labels:
      None

      Description

      After purging the Metrics Collector's database the following error message appears in the logs:

      2021-01-15 11:30:48,921 ERROR org.apache.ambari.metrics.core.timeline.discovery.TimelineMetricMetadataManager: TimelineMetricMetadataKey is null for : [-1, -1, -64, -112, -85, -17, 39, -84, -121, 19, -118, -36, -104, -21, -7, 110, 61, -97, -56, 10]

       

      The following steps were used to purge the database:

      • Get some configuration values:
      • Get hbase.rootdirhbase.rootdir = /user/ams/hbase
      • Get hbase.tmp.dirhbase.tmp.dir = /var/lib/ambari-metrics-collector/hbase-tmp
      • Get hbase.zookeeper.property.datadirhbase.zookeeper.property.datadir = ${hbase.tmp.dir}/zookeeper
      • Get phoenix.spool.directoryphoenix.spool.directory = ${hbase.tmp.dir}/phoenix-spool
      • Get ‘ZooKeeper Znode Parent’. ZooKeeper Znode Parent = /ams-hbase-unsecure
      • Stop AMS and set it to Maintenance Mode
      • Remove the HBase database
      • su hdfs
      • hdfs dfs -ls /user/ams/hbase
      • hdfs dfs -rm -r /user/ams/hbase
      • exit (back to root from hdfs user)
      • Remove the AMS Zookeeper data on both Collector Nodes
      • su ams
      • ls /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/
      • rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/*
      • Remove any Phoenix spool files on both Collector Nodes
      • ls /var/lib/ambari-metrics-collector/hbase-tmp/phoenix-spool/
      • rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/phoenix-spool/*
      • exit (back to root from ams user)
      • Connect to the cluster zookeeper instance and delete the ‘ZooKeeper Znode Parent’/meta-region-server node
      • /usr/hdp/current/zookeeper-client/bin/zkCli.sh -server localhost:2181
      • [zk: localhost:2181(CONNECTED) 0]
      • rmr /ams-hbase-unsecure/meta-region-server
      • quit
      • Restart AMS

       

        Attachments

          Activity

            People

            • Assignee:
              payert Tamas Payer
              Reporter:
              payert Tamas Payer
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: