[FLINK-24508] Examples: add ThrottledMapper, a walkaround for deadlock under heavy iteration feedback load

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.14.1
• Component/s: Examples

Environment

N/A

• Assignee: Unassigned
• Reporter: LI Zhennan
• Created: 11/Oct/21 14:40
• Updated: 11/Oct/21 14:51

Description

Hi,

Flink has a deadlock case and can be caused by the combination of Flink's backpressure mechanism with iterations (more likely when there is heavy feedback load).

There is a proposal(FLIP-15) to resolve this issue, but it seems to be abondoned: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=66853132

I propose to add `ThrottledMapper` into `flink-examples` directory of Flink repo, to serve as a walkaround before the deadlock issue gets resolved: https://gist.github.com/nanmu42/53fe23c3f15c777291108a7f4df62b9a

I am considering to add it into https://github.com/apache/flink/tree/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/utils

I am looking for committer support.

If it is a ok, I would like to contribute.

Thanks for your attention.

related issues:

• https://mail-archives.apache.org/mod_mbox/flink-user/201704.mbox/[e-mail]%3E
• https://issues.apache.org/jira/browse/FLINK-2497?jql=project%20%3D%20FLINK%20AND%20resolution%20%3D%20Unresolved%20AND%20text%20~%20%22deadlock%20iteration%22%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC