[KYLIN-5069] Remove find-hive-dependency.sh in Kylin 4

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v4.0.0
• Fix Version/s: v4.0.1
• Skill Level: Committer(Medium) - This is for regular contributors/committers
• Assignee: Yaqian Zhang
• Reporter: Xiaoxiang Yu
• Created: 30/Aug/21 08:12
• Updated: 19/Apr/22 02:29
• Resolved: 09/Nov/21 09:17

Description

In kylin.sh, Kylin introduces a lot of jars into Kylin instance's class path, via find-hive-dependency.sh and "hadoop classpath". So class conflict issues happened often, and most of them are related with hive jars. I think it is better to remove 'find-hive-dependency.sh' and use  SparkSession to fetch Hive meta.

 

Following is part of script kylin.sh :

tomcat_classpath=${tomcat_root}/bin/bootstrap.jar:${tomcat_root}/bin/tomcat-juli.jar:${tomcat_root}/lib/* export KYLIN_TOMCAT_CLASSPATH=${tomcat_classpath}:${KYLIN_HOME}/conf:${KYLIN_HOME}/lib/*:${KYLIN_HOME}/ext/*:${hadoop_dependencies}:${flink_dependency}

 

Some exception looks like this:

Issue Links

• relates to: KYLIN-5073 kylin can't start because `Not running on Jetty, JSR-356 support unavailable`
• relates to: KYLIN-5081 Kylin Can't read datasource from hive-3.1.2
• relates to: KYLIN-5084 Kylin4.0.0 is incompatible with HDP-3.1.5`s Hive-3.1.0