[SPARK-10530] Kill other task attempts when one taskattempt belonging the same task is succeeded in speculation

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Component/s: Scheduler, Spark Core
• Assignee: Devaraj Kavali
• Reporter: Jeff Zhang
• Created: 10/Sep/15 05:08
• Updated: 30/Aug/16 02:38
• Resolved: 30/May/16 21:30

Description

Currently when speculation is enabled, other task attempts won't be killed if one task attempt in the same task is succeeded, it is not resource efficient, it would be better to kill other task attempts when one taskattempt belonging the same task is succeeded in speculation.

Issue Links

• duplicates: SPARK-13965 TaskSetManager should kill the other running task attempts if any one task attempt succeeds for the same task
• is related to: SPARK-17300 ClosedChannelException caused by missing block manager when speculative tasks are killed
• links to: [Github] Pull Request #8683 (zjffdu)
• links to: [Github] Pull Request #11996 (devaraj-kavali)