[KYLIN-5084] Kylin4.0.0 is incompatible with HDP-3.1.5`s Hive-3.1.0

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Workaround
• Affects Version/s: v4.0.0
• Fix Version/s: v4.0.1
• Component/s: Metadata
• Environment: HDP-3.1.5+Hadoop-3.1.1+Spark-2.3.0+Hive-3.1.0
• Assignee: mukvin
• Reporter: lixinzhao
• Created: 06/Sep/21 06:56
• Updated: 06/Jan/22 06:48
• Resolved: 06/Sep/21 09:15

Description

The following error message is displayed during Hive`s table synchronization：

org.apache.kylin.rest.exception.InternalErrorException: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(Lorg/apache/hadoop/hive/conf/HiveConf;)V
at org.apache.kylin.rest.controller.TableController.showHiveDatabases(TableController.java:201) ~[kylin-server-base-4.0.0.jar:4.0.0] 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_291]
 ...............
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(Lorg/apache/hadoop/hive/conf/HiveConf;)V 
at org.apache.kylin.source.hive.HiveMetaStoreClientFactory.getHiveMetaStoreClient(HiveMetaStoreClientFactory.java:47) ~[kylin-source-hive-4.0.0.jar:4.0.0] 
at org.apache.kylin.source.hive.CLIHiveClient.getMetaStoreClient(CLIHiveClient.java:164) ~[kylin-source-hive-4.0.0.jar:4.0.0] 
at org.apache.kylin.source.hive.CLIHiveClient.getHiveDbNames(CLIHiveClient.java:118) ~[kylin-source-hive-4.0.0.jar:4.0.0]

I found the method named getHiveMetaStoreClient of org.apache.kylin.source.hive.HiveMetaStoreClientFactory is unbefitting,because the HiveMetaStoreClient`s init methos of hive3.1.2 no longer accepts parameters of type HiveConf but Configuration.

Issue Links

• is related to: KYLIN-5069 Remove find-hive-dependency.sh in Kylin 4
• relates to: KYLIN-5081 Kylin Can't read datasource from hive-3.1.2