[HIVE-24068] Add re-execution plugin for handling DAG submission and unmanaged AM failures

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Target Version/s: 4.0.0
• Assignee: Prasanth Jayachandran
• Reporter: Prasanth Jayachandran
• Created: 24/Aug/20 21:33
• Updated: 17/Nov/22 08:47
• Resolved: 26/Aug/20 17:48

Description

DAG submission failure can also happen in environments where AM container died causing DNS issues. DAG submissions are safe to retry as the DAG hasn't started execution yet. There are retries at getSession and submitDAG level individually but some submitDAG failure has to retry getSession as well as AM could be unreachable, this can be handled in re-execution plugin.

There is already AM loss retry execution plugin but it only handles managed AMs. It can be extended to handle unmanaged AMs as well.

Issue Links

• links to: GitHub Pull Request #1428