[KYLIN-1089] Kylin failed to run on CDH with HBase 1.0

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Workaround
• Affects Version/s: 1.x-HBase1.1.3
• Component/s: Environment
• Assignee: Shao Feng Shi
• Reporter: Shao Feng Shi
• Created: 20/Oct/15 07:56
• Updated: 11/Nov/16 00:52
• Resolved: 11/Nov/16 00:52

Description

Clone kylin "1.x-HBase1.x" branch and then make a binary package, upload to a CDH environment like CDH 5.4.2 quickstart vm, "kylin.sh start" will throw exception as below:

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'aclService' defined in file [/home/cloudera/kylin-1.1-incubating-SNAPSHOT/tomcat/webapps/kylin/WEB-INF/classes/org/apache/kylin/rest/service/AclService.class]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.apache.kylin.rest.service.AclService]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)Lorg/apache/hadoop/hbase/HTableDescriptor;
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:997)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:943)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
        at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:322)
        ... 116 more
Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.apache.kylin.rest.service.AclService]: Constructor threw exception; nested exception is java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)Lorg/apache/hadoop/hbase/HTableDescriptor;
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:162)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:76)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:990)
        ... 124 more
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.HTableDescriptor.addFamily(Lorg/apache/hadoop/hbase/HColumnDescriptor;)Lorg/apache/hadoop/hbase/HTableDescriptor;
        at org.apache.kylin.common.persistence.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:113)
        at org.apache.kylin.common.persistence.HBaseConnection.createHTableIfNeeded(HBaseConnection.java:86)
        at org.apache.kylin.rest.service.AclService.<init>(AclService.java:127)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147)
        ... 126 more

The root cause is, CDH reverts several Apache HBase 1.0 commits, for backward compatibility; While Kylin 1.x-HBase1.x is compiled with Apache HBase 1.1 API, which is not compitible with CDH; More readings about this problem:
https://issues.cask.co/browse/TEPHRA-53
https://groups.google.com/a/cloudera.org/forum/#!topic/cdh-user/KaGc7Ve7egs

Even if change "hbase-hadoop2.version" in kylin's pom.xml to "1.0.0-cdh5.4.2", it wouldn't compile as some classes like "org.apache.hadoop.hbase.regionserver.ScannerContext.java" doesn't exist in HBase 1.0;

Suggestion:

• if for trial, suggest user to run Kylin on an opensource HBase 1.1 installation, or choose HDP 2.3 sandbox vm;
• otherwise, please hold on to warit for CDH's future release;

Issue Links

• is duplicated by: KYLIN-1748 Apache Kylin - apache-kylin-1.5.2-HBase1.x-bin.tar.gz installation issue