[HUDI-7298] Write bad records to error table in more cases instead of failing stream

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.15.0, 1.0.0
• Component/s: deltastreamer, spark
• Labels: pull-request-available
• Assignee: Jonathan Vexler
• Reporter: Jonathan Vexler
• Created: 16/Jan/24 00:46
• Updated: 07/Jun/24 01:12
• Resolved: 25/Jan/24 04:11

Description

If no transformer is used, but schema provider is used, records with the incorrect schema will not be detected and will fail the stream during HoodieRecord creation. Additionally, during keygeneration the stream can crash if required fields are null.

Issue Links

• links to: GitHub Pull Request #10500