[YARN-8811] Support Container Storage Interface (CSI) in YARN

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: CSI
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 21/Sep/18 05:54
• Updated: 26/Apr/21 07:22

Description

The Container Storage Interface (CSI) is a vendor neutral interface to bridge Container Orchestrators and Storage Providers. With the adoption of CSI in YARN, it will be easier to integrate 3rd party storage systems, and provide the ability to attach persistent volumes for stateful applications.

Issue Links

• depends upon: YARN-8836 Add tags and attributes in resource definition

Sub-Tasks

• [YARN-8811] Add CSI java-based client library: Resolved
• [CSI] Extend service spec to allow setting resource attributes: Resolved
• [CSI] Add volume manager that manages CSI volume lifecycle: Resolved
• [CSI] Add volume as a top-level attribute in service spec: Open
• [CSI] CSI driver adaptor module support in NodeManager: Resolved
• [CSI] Ignore volume resource in resource calculators based on tags: Resolved
• [CSI] Add ability to publish/unpublish volumes on node managers: Resolved
• [CSI] CSI jar file should not bundle third party dependencies: Resolved
• [CSI] YARN service fail to launch due to CSI changes: Resolved
• [CSI] Run csi-driver-adaptor as aux service: Resolved
• [CSI] Upgrade CSI proto to v1.0: Patch Available