[SPARK-37893] Fix flaky test: AdaptiveQueryExecSuite with Scala 2.13

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: Tests
• Assignee: Yang Jie
• Reporter: Yang Jie
• Created: 13/Jan/22 08:19
• Updated: 14/Jan/22 02:57
• Resolved: 14/Jan/22 02:57

Description

Use maven test `AdaptiveQueryExecSuite` with Scala-2.13, the following exceptions will occur with a very small probability:

AdaptiveQueryExecSuite

- Logging plan changes for AQE *** FAILED ***
  java.util.ConcurrentModificationException: mutation occurred during iteration
  at scala.collection.mutable.MutationTracker$.checkMutations(MutationTracker.scala:43)
  at scala.collection.mutable.CheckedIndexedSeqView$CheckedIterator.hasNext(CheckedIndexedSeqView.scala:47)
  at scala.collection.StrictOptimizedIterableOps.filterImpl(StrictOptimizedIterableOps.scala:225)
  at scala.collection.StrictOptimizedIterableOps.filterImpl$(StrictOptimizedIterableOps.scala:222)
  at scala.collection.mutable.ArrayBuffer.filterImpl(ArrayBuffer.scala:43)
  at scala.collection.StrictOptimizedIterableOps.filterNot(StrictOptimizedIterableOps.scala:220)
  at scala.collection.StrictOptimizedIterableOps.filterNot$(StrictOptimizedIterableOps.scala:220)
  at scala.collection.mutable.ArrayBuffer.filterNot(ArrayBuffer.scala:43)
  at org.apache.spark.SparkFunSuite$LogAppender.loggingEvents(SparkFunSuite.scala:288)
  at org.apache.spark.sql.execution.adaptive.AdaptiveQueryExecSuite.$anonfun$new$152(AdaptiveQueryExecSuite.scal

 

 

Issue Links

• links to: [Github] Pull Request #35190 (LuciferYang)