[SPARK-19594] StreamingQueryListener fails to handle QueryTerminatedEvent if more then one listeners exists

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.1, 2.2.0
• Component/s: Structured Streaming
• Assignee: Eyal Zituny
• Reporter: Eyal Zituny
• Created: 14/Feb/17 12:34
• Updated: 26/Feb/17 23:59
• Resolved: 26/Feb/17 23:58

Description

reproduce:
*create a spark session
*add multiple streaming query listeners
*create a simple query
*stop the query
result -> only the first listener handle the QueryTerminatedEvent

this might happen because the query run id is being removed from activeQueryRunIds once the onQueryTerminated is called (StreamingQueryListenerBus:115)

Issue Links

• links to: [Github] Pull Request #16991 (eyalzit)