[SPARK-24294] Throw SparkException when OOM in BroadcastExchangeExec

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.4.0
• Component/s: SQL

Environment

scala-2.11.8

• Assignee: Jin Xing
• Reporter: Jin Xing
• Created: 16/May/18 07:47
• Updated: 23/May/18 20:12
• Resolved: 23/May/18 20:12

Description

When OutOfMemoryError thrown from BroadcastExchangeExec, scala.concurrent.Future will hit scala bug – https://github.com/scala/bug/issues/9554, and hang until future timeout:

We could wrap the OOM inside SparkException to resolve this issue.

Issue Links

• links to: [Github] Pull Request #21342 (jinxing64)