[HUDI-7678] Finalize the Merger APIs and make a plan for moving over all existing built-in, custom payloads.

• Type: Task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Story Points: 12
• Epic Link: RFC-46: Engine Native Record Payloads
• Assignee: Y Ethan Guo
• Reporter: Vinoth Chandar
• Created: 26/Apr/24 18:19
• Updated: 21/Oct/24 21:25
• Resolved: 21/Oct/24 21:25

Description

This ticket is to make sure that in Hudi 1.0 all existing built-in and custom payload classes should still work, providing the same functionality as 0.x releases.  They can be realized through either the new record merge API implementation, or still the existing payload class implementation through retrofit.

We'll fully migrate all exsiting payload logic to the new record merge implementation in HUDI-8401, i.e., with the move towards making partial updates a first class citizen, that does not need any special payloads/merges, we need to move the CDC payloads to all be transformers in Hudi Streamer and SQL write path. Along with migration instructions to users.