[HUDI-858] Allow multiple operations to be executed within a single commit

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5.3
• Component/s: writer-core
• Labels: pull-request-available
• Assignee: Balaji Varadarajan
• Reporter: Balaji Varadarajan
• Created: 01/May/20 08:45
• Updated: 23/May/20 12:26
• Resolved: 20/May/20 01:39

Description

There are users who had been directly using RDD APIs and have relied on a behavior in 0.4.x to allow multiple write operations (upsert/buk-insert/...) to be executed within a single commit. 

Given Hudi commit protocol, these are generally unsafe operations and user need to handle failure scenarios. It only works with COW table. Hudi 0.5.x had stopped this behavior.

Given the importance of supporting such cases for the user's migration to 0.5.x, we are proposing a safety flag (disabled by default) which will allow this old behavior.

Issue Links

• links to: GitHub Pull Request #1633