[SLIDER-77] use a window for tracking container failures

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Component/s: appmaster
• Sprint: Slider August #1, Slider August #2, Slider September #1
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 20/May/14 11:11
• Updated: 13/Aug/14 17:18
• Resolved: 13/Aug/14 17:18

Description

Use sliding windows and/or weighted moving averages to track container failures over time, and only react if many are failing in a short period.

What we do want to do here is react fast to a sudden series of failures, as well as look at average failure rates over time. I think separating startup failures from operational failures could help here. We don't want 5 failures in 5 minutes to be ignored just because everything worked well for the previous month

Issue Links

• is related to: SLIDER-309 add functional tests of sliding-window failure handling
• is related to: SLIDER-310 failure thresholds to be settable per-role
• relates to: YARN-611 Add an AM retry count reset window to YARN RM