[SPARK-25542] Flaky test: OpenHashMapSuite

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.0
• Component/s: Spark Core, Tests
• Assignee: L. C. Hsieh
• Reporter: Dongjoon Hyun
• Created: 26/Sep/18 06:43
• Updated: 24/Oct/18 18:12
• Resolved: 28/Sep/18 21:33

Description

• https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96585/testReport/org.apache.spark.util.collection/OpenHashMapSuite/_It_is_not_a_test_it_is_a_sbt_testing_SuiteSelector_/ (Sep 25, 2018 5:52:56 PM)

  org.apache.spark.util.collection.OpenHashMapSuite.(It is not a test it is a sbt.testing.SuiteSelector)
  
  Failing for the past 1 build (Since #96585 )
  Took 0 ms.
  Error Message
  sbt.ForkMain$ForkError: java.lang.OutOfMemoryError: Java heap space
  Stacktrace
  sbt.ForkMain$ForkError: sbt.ForkMain$ForkError: java.lang.OutOfMemoryError: Java heap space
  	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:117)
  	at scala.reflect.ManifestFactory$$anon$9.newArray(Manifest.scala:115)
  	at org.apache.spark.util.collection.OpenHashMap$$anonfun$1.apply$mcVI$sp(OpenHashMap.scala:159)
  	at org.apache.spark.util.collection.OpenHashSet.rehash(OpenHashSet.scala:234)
  	at org.apache.spark.util.collection.OpenHashSet.rehashIfNeeded(OpenHashSet.scala:171)
  	at org.apache.spark.util.collection.OpenHashMap$mcI$sp.update$mcI$sp(OpenHashMap.scala:86)
  	at org.apache.spark.util.collection.OpenHashMapSuite$$anonfun$17$$anonfun$apply$4.apply$mcVI$sp(OpenHashMapSuite.scala:192)
  	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
  	at org.apache.spark.util.collection.OpenHashMapSuite$$anonfun$17.apply(OpenHashMapSuite.scala:191)
  	at org.apache.spark.util.collection.OpenHashMapSuite$$anonfun$17.apply(OpenHashMapSuite.scala:188)
  	at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  	at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  	at org.scalatest.Transformer.apply(Transformer.scala:22)
  	at org.scalatest.Transformer.apply(Transformer.scala:20)
  	at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186)
  	at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:103)
  	at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:183)
  	at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196)
  	at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:196)
  	at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
  	at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:196)
  	at org.scalatest.FunSuite.runTest(FunSuite.scala:1560)
  	at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229)
  	at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:229)
  	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
  	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
  	at scala.collection.immutable.List.foreach(List.scala:392)
  	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
  	at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
  	at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
  	at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:229)
  	at org.scalatest.FunSuite.runTests(FunSuite.scala:1560)
  

Issue Links

• blocks: SPARK-24139 Fix Flaky Tests
• links to: [Github] Pull Request #22569 (viirya)
• links to: [Github] Pull Request #22569 (viirya)