Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4105

use spark engine to build cube on step Convert Cuboid Data to HFile failed

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: v2.4.0, v2.4.1, v2.5.0, v2.6.0, v2.6.1, v2.6.2
    • Fix Version/s: None
    • Component/s: Job Engine
    • Labels:
      None

      Description

      use spark engine to build cube on step Convert Cuboid Data to HFile。visit yarn logs has this exception:

      19/05/05 17:36:42 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 0.0 (TID 0, spslave4.bigdata.ly, executor 1): java.lang.ExceptionInInitializerError
      at org.apache.kylin.metadata.datatype.DataType.<clinit>(DataType.java:134)
      at java.io.ObjectStreamClass.hasStaticInitializer(Native Method)
      at java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1787)
      at java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:72)
      at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:253)
      at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:251)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:250)
      at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:611)
      at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
      at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1714)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
      at java.util.HashSet.readObject(HashSet.java:333)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1909)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
      at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
      at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
      at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
      at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
      at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:75)
      at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:114)
      at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:85)
      at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
      at org.apache.spark.scheduler.Task.run(Task.scala:99)
      at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:325)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.RuntimeException: fail to locate kylin.properties
      at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
      at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:354)
      at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:134)
      at org.apache.kylin.measure.MeasureTypeFactory.init(MeasureTypeFactory.java:120)
      at org.apache.kylin.measure.MeasureTypeFactory.<clinit>(MeasureTypeFactory.java:98)
      ... 71 more

        Attachments

          Activity

            People

            • Assignee:
              wangrupeng wangrupeng
              Reporter:
              dockj dockj
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: