[FLINK-9413] Tasks can fail with PartitionNotFoundException if consumer deployment takes too long

• Type: Bug
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.4.0, 1.5.0, 1.6.0
• Component/s: Runtime / Coordination
• Labels: auto-deprioritized-critical auto-deprioritized-major auto-deprioritized-minor pull-request-available
• Assignee: Unassigned
• Reporter: Till Rohrmann
• Created: 22/May/18 23:18
• Updated: 30/Dec/21 10:39

Description

Tasks can fail with a PartitionNotFoundException if the deployment of the producer takes too long. More specifically, if it takes longer than the taskmanager.network.request-backoff.max, then the Task will give up and fail.

The problem is that we calculate the InputGateDeploymentDescriptor for a consuming task once the producer has been assigned a slot but we do not wait until it is actually running. The problem should be fixed if we wait until the task is in state RUNNING before assigning the result partition to the consumer.

Issue Links

• links to: GitHub Pull Request #6103
• links to: GitHub Pull Request #6103