[AMBARI-22422] Ambari Metrics does not work with DateTieredStoreEngine

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.6.0
• Component/s: ambari-metrics

Environment

centos7

• Assignee: Unassigned
• Reporter: Zhanwei Wang
• Created: 11/Nov/17 07:38
• Updated: 17/Apr/18 01:57

Description

I build Ambari from source code with tag release-2.6.0 and setup a three node centos7 cluster for test.

With all default configurations, Ambari Metrics did not work and I found the following from HBase logs.

2017-11-11 15:07:56,478 INFO  [B.defaultRpcServer.handler=24,queue=0,port=40128] zookeeper.ZooKeeper: Initiating client connection, connectString=test3:61181 sessionTimeout=120000 watcher=hconnection-0x23f7d8350x0, quorum=test3:61181, baseZNode=/ams-hbase-unsecure
@                                                                                                    @                                                    at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
        at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Unable to load configured store engine 'org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine'
        at org.apache.hadoop.hbase.regionserver.StoreEngine.create(StoreEngine.java:128)
        at org.apache.hadoop.hbase.regionserver.HStore.<init>(HStore.java:271)
        at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:4958)
        at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:905)
        at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:902)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        ... 3 more
Caused by: java.lang.UnsupportedOperationException: Unable to find org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine
        at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:36)
        at org.apache.hadoop.hbase.regionserver.StoreEngine.create(StoreEngine.java:123)
        ... 10 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.regionserver.DateTieredStoreEngine
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:264)
        at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:32)
        ... 11 more

After adding the following to ams-site.xml. Ambari Metrics work normally. And no more Exception found in both collector and hbase logs.

    <property>
      <name>timeline.metrics.hbase.aggregate.table.compaction.policy.class</name>
      <value>org.apache.hadoop.hbase.regionserver.DefaultStoreEngine</value>
    </property>

From HBASE-19223, Date Tiered Compaction is only available starting in the 1.3 release.