[FLINK-10887] Add source watermark tracking to the JobMaster

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 1.8.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Jamie Grier
• Reporter: Jamie Grier
• Created: 14/Nov/18 21:41
• Updated: 28/Feb/19 14:00
• Resolved: 17/Feb/19 01:40

Description

We need to add a new RPC to the JobMaster such that the current watermark for every source sub-task can be reported and the current global minimum/maximum watermark can be retrieved so that each source can adjust their partition read rates in an attempt to keep sources roughly aligned in event time.

 

 

Issue Links

• links to: GitHub Pull Request #7099