Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
v2.5.1
-
None
-
kylin2.5.1
spark2.1.2
Description
Class is not registered in step "Extract Fact Table Distinct Columns", when use spark as build engine, error as follow:
com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Class is not registered: scala.reflect.ManifestFactory$$anon$2 Note: To register this class use: kryo.register(scala.reflect.ManifestFactory$$anon$2.class); Serialization trace: evidence$1 (org.apache.spark.util.collection.CompactBuffer) at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:101) at com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:518) at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:628) at org.apache.spark.serializer.KryoSerializationStream.writeObject(KryoSerializer.scala:207) at org.apache.spark.serializer.SerializationStream.writeValue(Serializer.scala:135) at org.apache.spark.storage.DiskBlockObjectWriter.write(DiskBlockObjectWriter.scala:239) at org.apache.spark.util.collection.ExternalAppendOnlyMap.org$apache$spark$util$collection$ExternalAppendOnlyMap$$spillMemoryIteratorToDisk(ExternalAppendOnlyMap.scala:227) at org.apache.spark.util.collection.ExternalAppendOnlyMap.spill(ExternalAppendOnlyMap.scala:186) at org.apache.spark.util.collection.ExternalAppendOnlyMap.spill(ExternalAppendOnlyMap.scala:56) at org.apache.spark.util.collection.Spillable.maybeSpill(Spillable.scala:97) at org.apache.spark.util.collection.ExternalAppendOnlyMap.insertAll(ExternalAppendOnlyMap.scala:160) at org.apache.spark.Aggregator.combineValuesByKey(Aggregator.scala:41) at org.apache.spark.shuffle.BlockStoreShuffleReader.read(BlockStoreShuffleReader.scala:91) at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:109) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:323) at org.apache.spark.rdd.RDD.iterator(RDD.scala:287) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87) at org.apache.spark.scheduler.Task.run(Task.scala:99) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: Class is not registered: scala.reflect.ManifestFactory$$anon$2 Note: To register this class use: kryo.register(scala.reflect.ManifestFactory$$anon$2.class); at com.esotericsoftware.kryo.Kryo.getRegistration(Kryo.java:488) at com.esotericsoftware.kryo.util.DefaultClassResolver.writeClass(DefaultClassResolver.java:97) at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:517) at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:76) ... 27 more
Attachments
Attachments
Issue Links
- links to