[FLINK-21364] piggyback finishedSplitIds in RequestSplitEvent

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.12.1
• Component/s: Connectors / Common
• Labels: auto-deprioritized-major auto-deprioritized-minor pull-request-available
• Assignee: Unassigned
• Reporter: Steven Zhen Wu
• Created: 11/Feb/21 16:39
• Updated: 24/Aug/23 10:35

Description

For some split assignment strategy, the enumerator/assigner needs to track the completed splits to advance watermark for event time alignment or rough ordering. Right now, `RequestSplitEvent` for FLIP-27 source doesn't support pass-along of the `finishedSplitIds` info and hence we have to create our own custom source event type for Iceberg source.

Here is the proposal of add such optional info to `RequestSplitEvent`.

public RequestSplitEvent(
    @Nullable String hostName, 
    @Nullable Collection<String> finishedSplitIds)

Issue Links

• links to: GitHub Pull Request #14929