[SPARK-6735] Provide options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0, 1.2.1, 1.3.0
• Fix Version/s: 2.0.0
• Component/s: Spark Submit, YARN
• Assignee: Saisai Shao
• Reporter: Twinkle Sachdeva
• Created: 07/Apr/15 06:23
• Updated: 28/Apr/16 17:39
• Resolved: 28/Apr/16 17:39

Description

Currently there is a setting (spark.yarn.max.executor.failures ) which tells maximum number of executor failures, after which Application fails.
For long running applications, user can require not to kill the application at all or will require such setting relative to a window duration. This improvement is ti provide such options to make maximum executor failure count ( which kills the application ) relative to a window duration or disable it.

Issue Links

• links to: [Github] Pull Request #5449 (twinkle-sachdeva)
• links to: [Github] Pull Request #10241 (jerryshao)