[SPARK-43299] JVM Client throw StreamingQueryException when error handling is implemented

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 4.0.0
• Component/s: Connect, Structured Streaming
• Labels: pull-request-available
• Epic Link: Streaming Spark Connect
• Assignee: Yihong He
• Reporter: Wei Liu
• Created: 26/Apr/23 20:49
• Updated: 03/Jan/24 00:11
• Resolved: 09/Oct/23 23:41

Description

Currently the awaitTermination() method of connect's JVM client's StreamingQuery won't throw error when there is an exception. 

 

In Python connect this is directly handled by python client's error-handling framework but such is not existed in JVM client right now.

 

We should verify it works when JVM adds that

 

Issue Links

• links to: GitHub Pull Request #42859
• links to: GitHub Pull Request #44306