[HUDI-1522] Add a new pipeline for Flink writer

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 0.8.0
• Component/s: writer-core
• Labels: pull-request-available
• Assignee: Danny Chen
• Reporter: Danny Chen
• Created: 11/Jan/21 07:21
• Updated: 28/Jan/21 00:54
• Resolved: 28/Jan/21 00:54

Description

Firstly, we can avoid the singleton task operator InstantGeneratorOperator by implementing an operator coordinator for the write operator, the coordinator always starts the checkpoint first, it starts a new commit on a new checkpointing

See RFC-24 #step1 for details.

Issue Links

• links to: GitHub Pull Request #2430