[FLINK-20081] ExecutorNotifier should run handler in the main thread when receive an exception from the callable.

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.11.2
• Fix Version/s: 1.11.3, 1.12.0
• Component/s: Connectors / Common
• Labels: pull-request-available
• Assignee: Jiangjie Qin
• Reporter: Jiangjie Qin
• Created: 11/Nov/20 05:42
• Updated: 06/Dec/20 12:30
• Resolved: 22/Nov/20 22:53

Description

Currently the ExecutorNotifier runs the handler in the worker thread if there is an exception thrown from the callable. This breaks the threading model and prevents an exception from bubbling up to fail the job.

Another issue is that right now, when an exception bubbles up from the SourceCoordinator, the UncaughtExceptionHandler will call System.exit(-17) and kill the JM. This is too much. Instead, we should just fail the job to trigger a failover.

Issue Links

• blocks: FLINK-20157 SourceCoordinatorProvider kills JobManager with IllegalStateException on job submission
• blocks: FLINK-20114 Fix a few KafkaSource-related bugs
• relates to: FLINK-20157 SourceCoordinatorProvider kills JobManager with IllegalStateException on job submission
• relates to: FLINK-20261 Uncaught exception in ExecutorNotifier due to split assignment broken by failed task
• links to: GitHub Pull Request #14030