[SPARK-28247] Flaky test: "query without test harness" in ContinuousSuite

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 2.4.4, 3.0.0
• Component/s: Structured Streaming, Tests
• Assignee: Jungtaek Lim
• Reporter: Jungtaek Lim
• Created: 04/Jul/19 07:20
• Updated: 26/Apr/21 23:46
• Resolved: 13/Jul/19 18:17

Description

This test has failed a few times in some PRs, as well as easy to reproduce locally. Example of a failure:

 [info] - query without test harness *** FAILED *** (2 seconds, 931 milliseconds)
[info]   scala.Predef.Set.apply[Int](0, 1, 2, 3).map[org.apache.spark.sql.Row, scala.collection.immutable.Set[org.apache.spark.sql.Row]](((x$3: Int) => org.apache.spark.sql.Row.apply(x$3)))(immutable.this.Set.canBuildFrom[org.apache.spark.sql.Row]).subsetOf(scala.Predef.refArrayOps[org.apache.spark.sql.Row](results).toSet[org.apache.spark.sql.Row]) was false
(ContinuousSuite.scala:226)

Issue Links

• duplicates: SPARK-24285 Flaky test: ContinuousSuite.query without test harness
• links to: [Github] Pull Request #32316 (zsxwing)
• links to: GitHub Pull Request #25048
• links to: GitHub Pull Request #25154