[KYLIN-3685] AWS Glue Catalog Not Supported

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Affects Version/s: v2.5.0
• Component/s: Integration
• Assignee: Kaige Liu
• Reporter: Richard Jarvis
• Created: 13/Nov/18 16:08
• Updated: 11/May/20 11:57
• Resolved: 11/May/20 11:57

Description

I am trying to use Kylin on AWS (EMR 5.18.0).

I use AWS Glue as the catalog and as a result Kylin can't find the tables. 

I am able to see the schemas and tables in the GUI because I have set the AWS glue properties in hive-site.xml:

 

<property>

<name>hive.metastore.client.factory.class</name>    <value>com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory</value>

</property>

However, the job org.apache.kylin.source.hive.cardinality.HiveColumnCardinalityJob fails to find the tables (it's looking in the Hive metadata catalog instead of AWS Glue).

I think this is because Hive 1.2.1 is too old to support the client factory class.

 

Issue Links

• Blocked: KYLIN-4206 Build kylin on EMR 5.23. The kylin version is 2.6.4. When building the cube, the hive table cannot be found
• is fixed by: KYLIN-4206 Build kylin on EMR 5.23. The kylin version is 2.6.4. When building the cube, the hive table cannot be found