[SPARK-17096] Fix StreamingQueryListener to return message and stacktrace of actual exception

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: Structured Streaming
• Target Version/s: 2.0.1, 2.1.0
• Assignee: Tathagata Das
• Reporter: Tathagata Das
• Created: 16/Aug/16 22:55
• Updated: 01/Nov/16 22:15
• Resolved: 17/Aug/16 20:32

Description

Currently only the stacktrace of StreamingQueryException is returned through StreamingQueryListener, which is useless as it hides the actual exception's stacktrace.

For example, if there is a / by zero exception in a task, the QueryTerminated.stackTrace will have

org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runBatches(StreamExecution.scala:211)
org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:124)

Issue Links

• links to: [Github] Pull Request #14675 (tdas)