[HUDI-3112] KafkaConnect can not sync to Hive

• Type: Task
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.10.1
• Component/s: hive
• Labels: pull-request-available sev:critical
• Epic Link: Hudi Kafka Connect
• Assignee: Unassigned
• Reporter: cdmikechen
• Created: 28/Dec/21 06:13
• Updated: 12/Jan/22 01:59
• Resolved: 12/Jan/22 01:59

Description

Right now KafkaConnect can not sync to Hive.
KafkaConnect use org.apache.hudi.DataSourceUtils to build HiveSyncConfig now, but in DataSourceUtils class import some spark dependencies. So that KafkaConnect will fail because of the application of related classes.

Caused by: java.lang.ClassNotFoundException: org.apache.spark.sql.types.DataType
	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 66 more

Issue Links

• is related to: HUDI-3199 Hive sync config unification
• links to: GitHub Pull Request #4458