[YARN-5636] Support reserving resources on certain nodes for certain applications

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: scheduler
• Assignee: Unassigned
• Reporter: Tao Jie
• Created: 12/Sep/16 02:35
• Updated: 12/Dec/17 07:39

Description

We have met such circumstance:
We are trying to run storm&kafka on yarn by Slider, and Storm&Kafka writes data to local disk on node. If some containers or the application fails, we expect that those containers would restart on the same node as they run before, otherwise data written on local would lost.
For slider, it will trying to ensure restarted container on same nodes as before. However for yarn, resource may be assigned to other applications when former long-running application is down.
As a result we'd better to have a mechanism that reserve some resource for certain long-running applications on certain nodes for a period of time. Does it make sense?

Issue Links

• incorporates: YARN-5829 FS preemption should reserve a node before considering containers on it for preemption