[KYLIN-1910] Support Separate HBase Cluster with NN HA and Kerberos Authentication

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v1.5.3
• Fix Version/s: v1.5.4
• Component/s: Job Engine
• Assignee: Yerui Sun
• Reporter: Yerui Sun
• Created: 21/Jul/16 09:41
• Updated: 15/Sep/16 03:35
• Resolved: 18/Aug/16 23:33

Description

Since KYLIN-957, we've support separate hbase cluster deployment. However, it didn't support hbase cluster with NameNode HA and Kerberos Authentication perfectly.

The key point is the cube building MR job need to access two hdfs cluster, the main cluster stored source data and the hbase cluster stored the cube result data. The MR task couldn't access hbase cluster with HA qualified path, like hdfs://hbase-cluster:8020/path, because that couldn't found dfs.nameservices related configs of hbase cluster.
To solve this problem, we need to read the hbase cluster HA configs and set into job conf.

Another point is about authentication tokens. During job submitting, the Resource Manager try to renew all tokens, to ensure can keep tokens alive. The hdfs token of hbase cluster renewing would cause exception, because that RM couldn't found the hbase cluster HA configs.
This problem can be resolved with YARN-3021 support.