[FLINK-17044] FLIP-108: Add GPU support in Flink

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Runtime / Coordination
• Assignee: Yangze Guo
• Reporter: Yangze Guo
• Created: 08/Apr/20 06:16
• Updated: 30/Nov/21 20:37
• Resolved: 18/Jun/20 10:23

Description

With widespread advances in machine learning (or deep learning), more and more enterprises are beginning to incorporate ML models across a number of products. Supporting the ML scenarios is one of Flink’s roadmap targets. GPU is widely used as the accelerator by people from the ML community. It is necessary to add GPU support. 

Currently, Flink only supports to request GPU resource in Mesos integration while most users and enterprises deploying Flink on Yarn/Kubernetes or Standalone mode. Thus, we propose to add GPU support in Flink. As a first step, we propose to:

• Enable user to configure the GPU cores per task executor and forward such requirements to the external resource managers (for Kubernetes/Yarn/Mesos setups).
• Provide information of available GPU resources to operators.

Issue Links

• relates to: FLINK-17795 Add an example to show how to leverage GPU resources
• mentioned in: Page Loading...