[TEZ-3758] Vertex can hang in RUNNING state when two task attempts finish very closely and have retroactive failures

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.1, 0.9.0
• Fix Version/s: 0.9.0
• Assignee: Kuhu Shukla
• Reporter: Kuhu Shukla
• Created: 09/Jun/17 20:28
• Updated: 22/Aug/17 00:02
• Resolved: 22/Jun/17 15:44

Description

A vertex's count of what tasks are done can go off in a case where two task attempts finish very closely, say within a millisecond of each other. We had a case where this task, which was marked successful, never scheduled another attempt upon getting a retroactive failure since it thought it had one uncompleted task attempt already. This is because the attempt that finished 1 ms later transitioned to SUCCEEDED but we don't take any action on the taskAttempStatus data structure and it stays false. This JIRA will attempt to solve that race.