[FLINK-14569] use flink to deal with batch task,when parallelism granter than 1, always 1 or 2 records loss

• Type: Bug
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.7.2
• Component/s: API / DataSet
• Labels: auto-deprioritized-critical auto-deprioritized-major auto-deprioritized-minor

Environment

hadoop:2.8.5

flink: 1.7.1

node machines: 3

deploy: yarn

• Assignee: Unassigned
• Reporter: kk
• Created: 30/Oct/19 08:15
• Updated: 20/Dec/21 10:39

Description

when flink read from hdfs file and set the parallelism>1, occasionally， 1  or 2 records  lose,  however the hdfs file bigger or smaller.