[SPARK-41072] Convert the internal error about failed stream to user-facing error

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 09/Nov/22 06:29
• Updated: 15/Nov/22 05:45
• Resolved: 15/Nov/22 05:45

Description

Assign an error class to the following internal error since it is an user-facing error:

java.lang.Exception: org.apache.spark.sql.streaming.StreamingQueryException: Query cloudtrail_pipeline [id = 5a3758c3-3b3a-47ff-843a-23292cde3b4f, runId = c1a90694-daa2-4929-b749-82b8a43fa2b1] terminated with exception: [INTERNAL_ERROR] Execution of the stream cloudtrail_pipeline failed. Please, fill a bug report in, and provide the full stack trace. 2 at org.apache.spark.sql.execution.streaming.StreamExecution.org$apache$spark$sql$execution$streaming$StreamExecution$$runStream(StreamExecution.scala:403) 3 at org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.$anonfun$run$4(StreamExecution.scala:269) 4 at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) 5 at com.databricks.unity.EmptyHandle$.runWithAndClose(UCSHandle.scala:42) 6 at org.apache.spark.sql.execution.streaming.StreamExecution$$anon$1.run(StreamExecution.scala:269) 7Caused by: java.lang.Exception: org.apache.spark.SparkException: [INTERNAL_ERROR] Execution of the stream cloudtrail_pipeline failed. Please, fill a bug report in, and provide the full stack trace. 8 at 

Issue Links

• links to: [Github] Pull Request #38623 (MaxGekk)
• links to: [Github] Pull Request #38629 (MaxGekk)