[HUDI-5514] Add support for auto generation of record keys for Hudi

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Component/s: writer-core
• Labels: pull-request-available release-0.14.0-blocker
• Epic Link: Primary Key-less Model
• Assignee: Danny Chen
• Reporter: sivabalan narayanan
• Created: 08/Jan/23 20:05
• Updated: 04/Aug/23 06:57
• Resolved: 09/May/23 12:34

Description

Hudi has a requirement to set record keys for any given table. But for some use-cases like ingesting log events, users may not have any column that can act as a primary key field. So, would be good to expose an auto generation record keys internally within hudi for such immutable use-cases. 

 

 

Issue Links

• duplicates: HUDI-5535 Add support for keyless for all keygens(non partitioned, timestamp based key gen)
• duplicates: HUDI-5571 Add support for keyless for all keygens(non partitioned, timestamp based key gen) row writer
• duplicates: HUDI-5586 Add support to auto generation of record keys for SimpleKeyGen
• duplicates: HUDI-5604 Test auto generation of record keys w/ sparks-sql
• is duplicated by: HUDI-5532 Add a KeyGenerator to support a Keyless workflow
• links to: GitHub Pull Request #7622
• links to: GitHub Pull Request #7640
• links to: GitHub Pull Request #7987
• links to: GitHub Pull Request #8107
• links to: GitHub Pull Request #8389
• links to: GitHub Pull Request #8697