[SPARK-24388] EventLoop's run method don't handle fatal error, causes driver hang forever

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.3.0
• Component/s: Scheduler, Spark Core
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: YE
• Created: 25/May/18 07:56
• Updated: 25/May/21 01:52
• Resolved: 25/May/21 01:41

Description

Once a fatal error(such as NoSuchMethodError) happens during `onReceive(event)`, the eventThread thread will exist. However the eventQueue is still accepting events. The whole spark application will hang forever.

 

cc zsxwing XuanYuan