[SLIDER-743] Include node failure history when choosing placement hints

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Slider 0.60
• Fix Version/s: Slider 0.70
• Component/s: appmaster
• Sprint: Slider Jan #2
• Assignee: Sumit Mohanty
• Reporter: Steve Loughran
• Created: 08/Jan/15 17:12
• Updated: 31/Jan/15 00:55
• Resolved: 31/Jan/15 00:55

Description

Use the (collected) history on per-node failures to tune placement requests, so that even if there's a history of components running there, if its been failing recently then it's not asked for

1. Add a "recent failures" field to the node entries, alongside the "total failures" counter
2. the scheduled failure count resetter will set that field to zero,alongside the component failures
3. When Slider has to request a new container, if a node has a recent failure count above some threshold, it will not be requested, even if it is a past location of that component.

When the placement policy is STRICT, node failure history will not be taken into account

Issue Links

• is duplicated by: SLIDER-758 Slider placement requests to skip unreliable nodes