[FLINK-19286] Improve pipelined region scheduling performance

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.12.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Zhu Zhu
• Reporter: Zhu Zhu
• Created: 18/Sep/20 05:47
• Updated: 19/Feb/21 07:24
• Resolved: 23/Sep/20 16:41

Description

In my recent TPCDS benchmark, pipelined region scheduling is slower than lazy-from-sources scheduling.
The regression is due to some suboptimal implementation of PipelinedRegionSchedulingStrategy, including:
1. topologically sorting of vertices to deploy
2. unnecessary O(V) loop when sorting an empty set of regions

After improving these implementations, pipelined region scheduling turned to be 10% faster in the previous benchmark setup.

Issue Links

• links to: GitHub Pull Request #13422