[YARN-2139] [Umbrella] Support for Disk as a Resource in YARN

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Wei Yan
• Created: 10/Jun/14 00:02
• Updated: 29/Sep/16 02:30

Description

YARN should consider disk as another resource for (1) scheduling tasks on nodes, (2) isolation at runtime, (3) spindle locality.

Issue Links

• is duplicated by: YARN-2791 Add Disk as a resource for scheduling
• is related to: YARN-2817 Disk drive as a resource in YARN
• is related to: YARN-5683 Support specifying storage type for per-application local dirs
• is related to: YARN-4276 Per-application-type local dirs for NodeManager
• relates to: YARN-896 Roll up for long-lived services in YARN
• relates to: YARN-3926 [Umbrella] Extend the YARN resource model for easier resource-type management and profiles

Sub-Tasks

• Avoid over-allocation of disk resources: Open
• NodeManager: Add cgroups support for disk I/O isolation: Resolved
• FairScheduler: Add disk I/O resource to the DRF implementation: Open
• MRAppMaster: Add support for disk I/O request: Open
• DistributedShell: Add support for disk I/O request: Open
• YarnClient: Add support for disk I/O resource/request information: Open
• WebUI & Metrics: Add disk I/O resource information to the web ui and metrics: Open
• CapacityScheduler: Add disk I/O resource to DRF: Open
• Revisit Disk IO configs: Open
• Add Disk as a resource for scheduling: Reopened