[OOZIE-3610] Need to improve Oozie job handling if oozie launcher Map is killed

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Shubham
• Created: 15/Sep/20 15:57
• Updated: 15/Sep/20 15:58

Description

 

• Due to Yarn Pre-emption, Yarn is killing Map task, which results in new Spark job. So we have duplicate job.

• We were able to reproduce issue after killing running Oozie launcher Map task.

• To fix above issue, we applied OOZIE-2194, which kills running Spark Job on map task kill(launcher job) and launch new one.

• If existing job is killed, we are wasting progress made by existing running job and it will again start from beginning

Issue Links

• relates to: OOZIE-2194 oozie job -kill doesn't work with spark action