[HUDI-7880] Support extraMetadata in Spark SQL Insert Into

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: spark-sql
• Labels: pull-request-available
• Assignee: 董可伦
• Reporter: 董可伦

Reviewers

Danny Chen

• Created: 14/Jun/24 05:50
• Updated: 16/Jun/24 00:16
• Resolved: 16/Jun/24 00:16

Description

Users want to implement checkpoints similar to those in Hudi DeltaStreamer. DeltaStreamer is implemented by saving values to extrametadata in a commit file, with the key deltastreamer.checkpoint.key. We can achieve this in Spark Client by configuring the parameter `house. datasource. write. commonmeta. key. prefix`, but in Spark SQL, it is restricted that the prefix of the configuration parameter must be `hoodie.`

Issue Links

• links to: GitHub Pull Request #11458