[SPARK-18280] Potential deadlock in `StandaloneSchedulerBackend.dead`

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.2, 2.0.0, 2.0.1
• Fix Version/s: 2.0.3, 2.1.0
• Component/s: Spark Core
• Assignee: Shixiong Zhu
• Reporter: Shixiong Zhu
• Created: 04/Nov/16 22:12
• Updated: 08/Dec/16 23:02
• Resolved: 08/Nov/16 21:19

Description

"StandaloneSchedulerBackend.dead" is called in a RPC thread, so it should not call "SparkContext.stop" in the same thread. "SparkContext.stop" will block until all RPC threads exit, if it's called inside a RPC thread, it will be dead-lock.

Issue Links

• links to: [Github] Pull Request #15775 (zsxwing)