[HUDI-1396] Bootstrap job via Hudi datasource hangs at the end of the spark-submit job

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Wenning Ding
• Created: 14/Nov/20 04:26
• Updated: 21/Jan/21 05:57
• Resolved: 21/Jan/21 05:57

Description

Bootstrap job via Hudi datasource hangs at the end of the spark-submit job

 

This issue is similar to https://issues.apache.org/jira/browse/HUDI-1230. Basically, HoodieWriteClient at https://github.com/apache/hudi/blob/release-0.6.0/hudi-spark/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala#L255 will not be closed and as a result, the corresponding timeline server will not stop at the end. Therefore the job hangs and never exits.

Issue Links

• links to: GitHub Pull Request #2253