[FLINK-4344] Implement new JobManager

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: Runtime / Coordination
• Labels: flip-6
• Assignee: Unassigned
• Reporter: Stephan Ewen
• Created: 09/Aug/16 22:20
• Updated: 28/Feb/19 14:13
• Resolved: 13/Jul/18 07:14

Description

This is the parent issue for the efforts to implement the JobManager changes based on FLIP-6 (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077)

Because of the breadth of changes, we should implement a new version of the JobManager (let's call it JobMaster) rather than updating the current JobManager. That will allow us to keep a working master branch.
At the point when the new cluster management is on par with the current implementation, we will drop the old JobManager and rename the JobMaster to JobManager.

Here is a sketch about the new design of the JobManager.

Issue Links

• blocks: FLINK-4319 Rework Cluster Management (FLIP-6)
• depends upon: FLINK-4457 Make the ExecutionGraph independent of Akka
• is blocked by: FLINK-7871 SlotPool should release its unused slot to RM
• is blocked by: FLINK-7870 SlotPool should cancel the slot request to RM if not need any more.
• is blocked by: FLINK-4459 Introduce SlotProvider for Scheduler
• is broken by: FLINK-6434 There may be allocatedSlots leak in SlotPool
• requires: FLINK-4568 Replace ActorGateway in WebRuntimeMonitor