Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Cannot Reproduce
-
0.8.1
-
None
Description
when I use
org.apache.spark.serializer.KryoSerializer
as spark serializer, I got failed
INFO [2019-03-22 11:05:07,501] ({task-result-getter-3} Logging.scala[logInfo]:54) - Removed TaskSet 1.0, whose tasks have all completed, from pool INFO [2019-03-22 11:05:07,502] ({dag-scheduler-event-loop} Logging.scala[logInfo]:54) - Cancelling stage 1 INFO [2019-03-22 11:05:07,502] ({dag-scheduler-event-loop} Logging.scala[logInfo]:54) - ResultStage 1 (take at <console>:30) failed in 0.077 s due to Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 7, datanode-3, executor 1): java.io.EOFException at org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:248) at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$9.apply(TorrentBroadcast.scala:290) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1318) at org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:291) at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:225) at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1284) at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:206) at org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66) at org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66) at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96) at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:81) at org.apache.spark.scheduler.Task.run(Task.scala:100) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)