[SPARK-25027] LegacyAccumulatorWrapper test fails in Scala 2.12

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.4.0
• Component/s: Build, Spark Core, Tests
• Assignee: Sean R. Owen
• Reporter: Sean R. Owen
• Created: 04/Aug/18 22:11
• Updated: 06/Sep/18 14:15
• Resolved: 06/Sep/18 14:15

Description

I've found at least one test failure in the 2.12; am running further to see if there are more.

- LegacyAccumulatorWrapper with AccumulatorParam that has no equals/hashCode *** FAILED ***
java.io.NotSerializableException: org.scalatest.Assertions$AssertionsHelper
Serialization stack:
- object not serializable (class: org.scalatest.Assertions$AssertionsHelper, value: org.scalatest.Assertions$AssertionsHelper@1b27bfe7)
- field (class: org.scalatest.FunSuite, name: assertionsHelper, type: class org.scalatest.Assertions$AssertionsHelper)
- object (class org.apache.spark.util.AccumulatorV2Suite, AccumulatorV2Suite)
- field (class: org.apache.spark.util.AccumulatorV2Suite$$anon$1, name: $outer, type: class org.apache.spark.util.AccumulatorV2Suite)
- object (class org.apache.spark.util.AccumulatorV2Suite$$anon$1, org.apache.spark.util.AccumulatorV2Suite$$anon$1@52d0064e)
- field (class: org.apache.spark.util.LegacyAccumulatorWrapper, name: param, type: interface org.apache.spark.AccumulableParam)
- object (class org.apache.spark.util.LegacyAccumulatorWrapper, LegacyAccumulatorWrapper(id: 358431, name: Some(test), value: org.apache.spark.util.AccumulatorV2Suite$MyData$1@7385bc7a))
at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:41)
at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:46)
at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:100)
at org.apache.spark.util.AccumulatorV2Suite.$anonfun$new$5(AccumulatorV2Suite.scala:182)
at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186)
...

Issue Links

• duplicates: SPARK-25029 Scala 2.12 issues: TaskNotSerializable and Janino "Two non-abstract methods ..." errors