[YARN-5468] Scheduling of long-running applications

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: capacityscheduler, fairscheduler
• Assignee: Konstantinos Karanasos
• Reporter: Konstantinos Karanasos
• Created: 02/Aug/16 23:16
• Updated: 12/May/17 22:23
• Resolved: 12/May/17 21:35

Description

This JIRA is about the scheduling of applications with long-running tasks.
It will include adding support to the YARN for a richer set of scheduling constraints (such as affinity, anti-affinity, cardinality and time constraints), and extending the schedulers to take them into account during placement of containers to nodes.
We plan to have both an online version that will accommodate such requests as they arrive, as well as a Long-running Application Planner that will make more global decisions by considering multiple applications at once.

Issue Links

• depends upon: YARN-6592 [Umbrella] Rich placement constraints in YARN

Sub-Tasks

• Add container tags to resource requests: Resolved
• Store container tags in ResourceManager: Resolved
• Expose API in ApplicationSubmissionContext to specify placement constraints: Resolved
• Add placement constraints to the resource requests: Resolved