[FLINK-27222] Execution history limit can lead to eviction of critical local-recovery information

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.15.0, 1.16.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 13/Apr/22 10:48
• Updated: 05/May/22 07:39
• Resolved: 15/Apr/22 07:21

Description

Local recovery relies on knowing the allocation id of the last deployment. To that end we iterate over all previous execution attempts and use the last assignedAllocationID, if any.
However, since the execution history is bounded (to, by default, 16 entries) this can lead this information being evicted.

In other words, with the default configuration (history limit = 16, restart delay = 1s) local recovery can only kick if the TM is restarted within 16 seconds.

We should decouple this information from the execution (history).

Issue Links

• is related to: FLINK-27127 Local recovery is not triggered on task manager process restart
• links to: GitHub Pull Request #19469