[HUDI-1369] Bootstrap datasource jobs from hanging via spark-submit

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 0.14.0
• Story Points: 2
• Epic Link: Improve Bootstrap
• Assignee: Ethan Guo
• Reporter: Wenning Ding
• Created: 04/Nov/20 07:17
• Updated: 23/May/23 04:16

Description

MOR table creation via Hudi datasource hangs at the end of the spark-submit job.

Looks like HoodieWriteClient at https://github.com/apache/hudi/blob/release-0.6.0/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L255 not being closed which does not stop the timeline server at the end, and as a result the job hangs and never exits.