[KYLIN-791] in the hadoop envirment by extracting tar files kylin does not work

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: v0.7.1
• Component/s: Environment
• Environment: CentOS6.5; kylin 0.7.1; CDH5.3(same errors in CDH5.1 and 5.2)
• Assignee: Hongbin Ma
• Reporter: blake.liu
• Created: 28/May/15 01:56
• Updated: 09/Aug/17 07:29
• Resolved: 07/Sep/15 02:14

Description

In the cloudera manager installation or yum installation, kylin0.7 runs well, but in the hadoop envirment by extracting tar files kylin can not normally start. After specifying hive_conf_path and hive_exec_path environment variable in find-hive-dependency.sh script ,kylin can start, but i got some errors when using "load hive table" and "build Cube action" features.

--------------------------------------------------------------------------------------------------
The following is the environment variable configuration

BASE_PATH=/opt/hadoop/soft

JAVA_HOME=$BASE_PATH/jdk1.7.0_75
JRE_HOME=$JAVA_HOME/jre
PATH=$JAVA_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

HADOOP_HOME=$BASE_PATH/hadoop-2.5.0-cdh5.3.0
HBASE_HOME=$BASE_PATH/hbase-0.98.6-cdh5.3.0
SPARK_HOME=$BASE_PATH/spark-1.2.0-cdh5.3.0
HIVE_HOME=$BASE_PATH/hive-0.13.1-cdh5.3.0
KYLIN_HOME=/opt/kylin-0.7.1-incubating-SNAPSHOT

PATH=$HADOOP_HOME/bin:$HBASE_HOME/bin:$SPARK_HOME/bin:$HIVE_HOME/bin:$PATH
export JAVA_HOME JRE_HOME HADOOP_HOME HBASE_HOME SPARK_HOME HIVE_HOME PATH CLASSPATH
--------------------------------------------------------------------------------------------------
The following is the error in kylin.log

[pool-7-thread-2]:[2015-05-21 10:29:27,822][ERROR][org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:109)] - error running Executable
java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/metadata/HiveStorageHandler
at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:146)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:94)
at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 16 more