[SPARK-17675] Add Blacklisting of Executors & Nodes within one TaskSet

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1.0
• Component/s: Scheduler, Spark Core
• Assignee: Imran Rashid
• Reporter: Imran Rashid
• Created: 27/Sep/16 02:03
• Updated: 17/May/20 17:47
• Resolved: 12/Oct/16 21:46

Description

This is a step along the way to SPARK-8425 – see the design doc on that jira for a complete discussion of blacklisting.

To enable incremental review, the first step proposed here is to expand the blacklisting within tasksets. In particular, this will enable blacklisting for

• (task, executor) pairs (this already exists via an undocumented config)
• (task, node)
• (taskset, executor)
• (taskset, node)

In particular, adding (task, node) is critical to making spark fault-tolerant of one-bad disk in a cluster, without requiring careful tuning of "spark.task.maxFailures". The other additions are also important to avoid many misleading task failures and long scheduling delays when there is one bad node on a large cluster.

Issue Links

• blocks: SPARK-8425 Add blacklist mechanism for task scheduling
• is duplicated by: SPARK-4681 Turn on executor level blacklisting by default
• links to: [Github] Pull Request #15249 (squito)