[SLIDER-799] AM to decide when to relax placement policy from specific host to rack/cluster

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Slider 0.70
• Fix Version/s: Slider 0.80
• Component/s: appmaster
• Sprint: Slider Feb #1, Slider April #1
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 27/Feb/15 23:17
• Updated: 26/Mar/15 18:02
• Resolved: 25/Mar/15 20:32

Description

If Slider asks for relaxed affinity, YARN only gives it ~1 second for free capacity to appear on a node before it falls back to non-local assignment. While this is OK for analytics throughput, it's suboptimal for placement of code such as HBase region servers.

AM needs to take charge of the placement and decide for itself when to convert from placed to relaxed.

Issue Links

• is depended upon by: SLIDER-611 Über-JIRA : placement phase 2
• relates to: YARN-3309 Capacity scheduler can wait a very long time for node locality