[FLINK-11391] Introduce ShuffleMaster in Job Master

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.9.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Andrey Zagrebin
• Reporter: Andrey Zagrebin
• Created: 18/Jan/19 13:00
• Updated: 23/Aug/19 07:58
• Resolved: 06/Jun/19 09:40

Description

• Implement PartitionShuffleDescriptor for covering necessary abstract info.
• Implement ShuffleDeploymentDecriptor generated from PartitionShuffleDecriptor.
• Define ShuffleMaster interface and create a simple implementation on JM side which relies on currently implemented NetworkEnviroment on TM side.
• Define ShuffleManager interface for creating ShuffleMaster.
• Introduce a Flink configuration option for ShuffleManager implementation. Default value for it could be <none> which serves as a feature flag at the moment to use current code paths.

Issue Links

• links to: GitHub Pull Request #7631
• links to: GitHub Pull Request #8362