[FLINK-33121] Failed precondition in JobExceptionsHandler due to concurrent global failures

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.18.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Panagiotis Garefalakis
• Reporter: Panagiotis Garefalakis
• Created: 20/Sep/23 05:54
• Updated: 28/Mar/24 17:06
• Resolved: 28/Mar/24 17:06

Description

We make the assumption that Global Failures (with null Task name) may only be RootExceptions and and Local/Task exception may be part of concurrent exceptions List (see JobExceptionsHandler#createRootExceptionInfo).
However, when the Adaptive scheduler is in a Restarting phase due to an existing failure (that is now the new Root) we can still, in rare occasions, capture new Global failures, violating this condition (with an assertion is thrown as part of assertLocalExceptionInfo) seeing something like:

The taskName must not be null for a non-global failure.  

We want to ignore Global failures while being in a Restarting phase on the Adaptive scheduler until we properly support multiple Global failures in the Exception History as part of https://issues.apache.org/jira/browse/FLINK-34922

Note: DefaultScheduler does not suffer from this issue as it treats failures directly as HistoryEntries (no conversion step)

Issue Links

• is fixed by: FLINK-34922 Exception History should support multiple Global failures
• is related to: FLINK-33565 The concurrentExceptions doesn't work
• is related to: FLINK-34922 Exception History should support multiple Global failures
• links to: GitHub Pull Request #23440