[FLINK-8717] Flink seems to deadlock due to buffer starvation when iterating

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 1.4.0
• Component/s: API / DataStream

Environment

Windows 10 Pro 64-bit

Core i7-6820HQ @ 2.7 GHz

16GB RAM

Flink 1.4

Scala client

Scala 2.11.7

 

• Assignee: Unassigned
• Reporter: Romain Revol
• Created: 20/Feb/18 15:13
• Updated: 28/Feb/19 11:14
• Resolved: 21/Feb/18 08:34

Description

We are encountering what looks like a deadlock of Flink in one of our jobs with an "iterate" in it.

I've reduced the job use case to the example in this gist : https://gist.github.com/rrevol/06ddfecd5f5ac7cbc67785b5d3a84dd4

Nothe that :

• varying the parallelism affects the rapidity of occurence of the deadlock, but it always occur
• varying MAX_LOOP_NB does affect the deadlock : the higher it is, the faster we encounter the deadlock. If MAX_LOOP_NB == 1, no deadlock. It consequently leads to think that it happens when the number of iterations reaches some threshold.

From the threadDump.txt, it looks like some starvation over buffer allocation, maybe backpressure has flaws on iterate, but I may be mistaking since I don't know well Flink internals.

Issue Links

• Is contained by: FLINK-5089 Introduce Loop functions and Enforce Nesting on Data Streams