[TEZ-746] DAG and Vertex commit should ensure that all tasks have completed before committing

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Bikas Saha
• Reporter: Bikas Saha
• Created: 21/Jan/14 03:11
• Updated: 07/May/19 23:04
• Resolved: 07/May/19 23:04

Description

There may be a race when a task (say a speculative task) is still running. Then the committer commits (say lists & cleans some files in the output dir). Then that task writes some temporary files in the output dir and gets killed. Those temp files may remain behind.

Issue Links

• is duplicated by: TEZ-2249 Wait for all task attempt finished before moving Task to finished state