[FLINK-13056] Optimize region failover performance on calculating vertices to restart

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.9.0, 1.10.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Zhu Zhu
• Reporter: Zhu Zhu
• Created: 02/Jul/19 09:34
• Updated: 03/Aug/22 03:44
• Resolved: 10/Mar/21 12:22

Description

Currently some region boundary structures are calculated each time of a region failover. This calculation can be heavy as its complexity goes up with execution edge count.

We tested it in a sample case with 8000 vertices and 16,000,000 edges. It takes ~2.0s to calculate vertices to restart.

(more details in https://docs.google.com/document/d/197Ou-01h2obvxq8viKqg4FnOnsykOEKxk3r5WrVBPuA/edit?usp=sharing)

That's why we'd propose to cache the region boundary structures to improve the region failover performance.

Issue Links

• duplicates: FLINK-21328 Optimize the initialization of DefaultExecutionTopology
• links to: GitHub Pull Request #9688