[SPARK-5835] Unit test causes java.io.FileNotFoundException on localhost for file "broadcast_1"

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 1.0.0
• Component/s: Tests
• Assignee: Unassigned
• Reporter: sam
• Created: 16/Feb/15 11:55
• Updated: 17/Feb/15 11:13
• Resolved: 17/Feb/15 11:13

Description

Note, I do not believe this is related to SPARK-2984 since I have speculative execution off (it's off by default in 1.0.0).

I intermittently get the following stack trace in my unit tests. I'm using specs2 and I have "sequential" in the tests (so should not be bumping into each other), and also I have `parallelExecution in Test := false` in my `build.sbt`.

This isn't a major showstopper, it just means our CI pipelines need some "retry" logic to workaround the erroring tests.

[error] Could not run test my.test.Class: org.apache.spark.SparkException: Job aborted due to stage failure: Task 4.0:0 failed 1 times, most recent failure: Exception failure in TID 6 on host localhost: java.io.FileNotFoundException: http://blar.blar.blar.blar:59528/broadcast_1
[error] sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1834)
[error] sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:90)
[error] sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1431)
[error] sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1429)
[error] java.security.AccessController.doPrivileged(Native Method)
[error] java.security.AccessController.doPrivileged(AccessController.java:713)
[error] sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1428)
[error] org.apache.spark.broadcast.HttpBroadcast$.read(HttpBroadcast.scala:196)
[error] org.apache.spark.broadcast.HttpBroadcast.readObject(HttpBroadcast.scala:89)
[error] sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] scala.collection.immutable.$colon$colon.readObject(List.scala:362)
[error] sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:483)
[error] java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1896)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1993)
[error] java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1918)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
[error] org.apache.spark.scheduler.ShuffleMapTask$.deserializeInfo(ShuffleMapTask.scala:63)
[error] org.apache.spark.scheduler.ShuffleMapTask.readExternal(ShuffleMapTask.scala:135)
[error] java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:1840)
[error] java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1799)
[error] java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
[error] java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
[error] org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:63)
[error] org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:85)
[error] org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:169)
[error] java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[error] java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[error] java.lang.Thread.run(Thread.java:745)
[error] Driver stacktrace: