[FLINK-14474] FLIP-67 Cluster partitions

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Runtime / Coordination
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 21/Oct/19 13:31
• Updated: 23/Oct/20 10:06
• Resolved: 16/Apr/20 09:12

Description

Umbrella issue for implementing cluster partitions, in accordance with FLIP-67.

Issue Links

• is duplicated by: FLINK-12419 Add support for consuming BLOCKING_PERSISTENT ResultPartition

Sub-Tasks

• Adjust TaskExecutor interface to support partition promotions: Closed
• Extend PartitionTracker to support partition promotions: Closed
• Extend partition tracking and safety net on TaskExecutor: Closed
• Send PartitionReport to ResourceManager via heartbeats: Closed
• Add partition management REST API endpoints: Closed
• Generalize TE->RM payload: Closed
• Capture partition meta info on TaskExecutor: Closed
• Separate PartitionTracker into generic and JM-specific versions: Closed
• JobMaster issues promote calls if job is successful: Closed
• Add total number of partitions to ResultPartitionDeploymentDescriptor: Closed
• Process ClusterPartitionReport on ResourceManager: Closed
• Extend TaskExecutor interface to support release of cluster partitions: Closed
• Rework TaskExecutorPartitionLifecycleTest: Closed