[AMBARI-21902] Ambari is setting "atlas.graph.storage.hostname" to hbase hostname, instead it should set the zookeeper quorum.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.5.0, 2.6.0
• Fix Version/s: 2.6.0
• Component/s: ambari-server
• Assignee: Ayub Pathan
• Reporter: Ayub Pathan
• Created: 07/Sep/17 10:51
• Updated: 13/Sep/17 09:45
• Resolved: 13/Sep/17 09:45

Description

Atlas is connecting to a host where zookeeper is not installed, thinking that there is zookeeper instance running on that host. Ambari is setting this property "atlas.graph.storage.hostname" to hbase hostname, instead it should set to zookeeper quorum. Since Hbase is managed by zookeeper, this setting should be fine.

Please note, this issue will not be seen if Hbase master and zookeeper are hosted on the same node.

To explain in detail:

• There is a configuration property(atlas.graph.storage.hostname) in Atlas which defines where Hbase storage is installed.
• This property is managed by Ambari and it is set to the hostname where HBase is installed.
• When Atlas starts, it initializes the connection with HBase using HBaseStoreManger class, which overrides “hbase.zookeeper.quorum” property value with “atlas.graph.storage.hostname” property value.
• If there is zookeeper node running on the same host as HBase host, then the HBase initialization will succeed, everything is fine.
• If there is no zookeeper node is running on the same host as HBase, then the above zookeeper connection issue can be observed, which will eventually result in Atlas failure.

To fix this, “atlas.graph.storage.hostname” config property in Atlas(managed by Ambari) can be set to the cluster zookeeper quorum value. Since HBase is managed by Zookeeper, this setting should be fine.

Fix can be done here: https://github.com/apache/ambari/blob/branch-2.6/ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java#L2734

2017-09-06 23:17:20,518 INFO  - [main:] ~ Instantiated HBase compatibility layer supporting runtime HBase version 1.1.2.2.6.3.0-61: com.thinkaurelius.titan.diskstorage.hbase.HBaseCompat1_1 (HBaseCompatLoader:69)
2017-09-06 23:17:20,560 INFO  - [main:] ~ Copied host list from root.storage.hostname to hbase.zookeeper.quorum: ctr-e134-1499953498516-130328-01-000003.hwx.site (HBaseStoreManager:320)
2017-09-06 23:17:20,666 INFO  - [main:] ~ Process identifier=hconnection-0x80b122b connecting to ZooKeeper ensemble=ctr-e134-1499953498516-130328-01-000003.hwx.site:2181 (RecoverableZooKeeper:120)

Issue Links

• links to: ReviewBoard