[SPARK-13965] TaskSetManager should kill the other running task attempts if any one task attempt succeeds for the same task

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.6.1
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Devaraj Kavali
• Created: 17/Mar/16 09:04
• Updated: 28/Mar/16 09:25
• Resolved: 28/Mar/16 09:25

Description

When we enable speculation, Driver would launch additional attempts for the same task if it founds that attempt is progressing slow compared to other tasks average progress and then there will be multiple task attempts in running state.

At present, if any one attempt gets succeeded others would be keep running (even they could run till the job completion) and cannot be given these slots to other tasks in same stage or in next stages.

We can kill these running task attempts when any other attempt gets succeeded and can be given the slots to run other tasks.

Issue Links

• is duplicated by: SPARK-10530 Kill other task attempts when one taskattempt belonging the same task is succeeded in speculation
• links to: [Github] Pull Request #11778 (devaraj-kavali)