[FLINK-25919] Sink V2 improvements and followups

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.16.0
• Component/s: API / DataStream, Connectors / Common
• Assignee: Unassigned
• Reporter: Fabian Paul
• Created: 02/Feb/22 09:10
• Updated: 02/Feb/22 09:10

Description

This is an umbrella ticket for know limitations and improvements we still want to do for the Sink V2 https://cwiki.apache.org/confluence/display/FLINK/FLIP-191%3A+Extend+unified+Sink+interface+to+support+small+file+compaction

Sub-Tasks

• Allow receiving updates of CommittableSummary: Open
• Support different input parallelism for preCommit topology: Open
• Add committer metrics to track the status of committables: Resolved
• Allow users to configure what to do with errors while committing transactions during recovery in KafkaSink: Open
• Separate SinkV2 and SinkV1Adapter tests: Closed
• Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable: Closed