[SPARK-28418] Flaky Test: pyspark.sql.tests.test_dataframe: test_query_execution_listener_on_collect

• Type: Test
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 2.3.4, 2.4.4, 3.0.0
• Component/s: PySpark, SQL
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 17/Jul/19 06:25
• Updated: 12/Dec/22 17:51
• Resolved: 19/Aug/19 03:28

Description

ERROR [0.164s]: test_query_execution_listener_on_collect (pyspark.sql.tests.test_dataframe.QueryExecutionListenerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/python/pyspark/sql/tests/test_dataframe.py", line 758, in test_query_execution_listener_on_collect
    "The callback from the query execution listener should be called after 'collect'")
AssertionError: The callback from the query execution listener should be called after 'collect'

Seems it can be failed due to not waiting events to be proceeded.

Issue Links

• links to: GitHub Pull Request #25177