[FLINK-21439] Adaptive Scheduler: Add support for exception history

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available reactive
• Release Note: add exception history for AdaptiveScheduler
• Assignee: Niklas Semmler
• Reporter: Matthias Pohl
• Created: 22/Feb/21 13:07
• Updated: 01/Mar/22 14:06
• Resolved: 17/Feb/22 21:42

Description

SchedulerNG.requestJob returns an ExecutionGraphInfo that was introduced in FLINK-21188. This ExecutionGraphInfo holds the information about the ArchivedExecutionGraph and exception history information. Currently, it's a list of ErrorInfos. This might change due to ongoing work in FLINK-21190 where we might introduced a wrapper class with more information on the failure.

The goal of this ticket is to implement the exception history for the AdaptiveScheduler, i.e. collecting the exceptions that caused restarts. This collection of failures should be forwarded through SchedulerNG.requestJob.

Issue Links

• causes: FLINK-26431 Release Testing: Exception history for the adaptive scheduler
• depends upon: FLINK-21075 FLIP-160: Adaptive scheduler
• is blocked by: FLINK-21190 Expose extended exception history through REST handler and adapt UI accordingly
• is related to: FLINK-6042 Display last n exceptions/causes for job restarts in Web UI
• relates to: FLINK-26289 Adaptive Scheduler: Exception history can not be queried by the REST API
• links to: GitHub Pull Request #15898
• links to: GitHub Pull Request #18689