[FLINK-10331] Reduce number of flush requests to the network stack

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0, 1.5.1, 1.5.2, 1.5.3, 1.6.0, 1.7.0
• Fix Version/s: 1.5.5, 1.6.2, 1.7.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Nico Kruber
• Reporter: Nico Kruber
• Created: 13/Sep/18 08:39
• Updated: 17/Oct/18 08:39
• Resolved: 19/Sep/18 19:35

Description

With the re-design of the record writer interaction with the result(sub)partitions, flush requests can currently pile up in these scenarios:

• a previous flush request has not been completely handled yet and/or is still enqueued or
• the network stack is still polling from this subpartition and doesn't need a new notification

These lead to increased notifications in low latency settings (low output flusher intervals) which can be avoided.

Issue Links

• links to: GitHub Pull Request #6692