2022-01-06 11:29:05,823 WARN  [pool-3-thread-1] util.NativeCodeLoader : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2022-01-06 11:29:06,072 INFO  [pool-3-thread-1] application.SparkApplication : Executor task org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob with args : {"distMetaUrl":"kylin_metadata@hdfs,path=hdfs://master:8020/kylin/kylin_metadata/PM01/job_tmp/05799056-b559-4f8c-8d7c-5db030229392-00/meta","submitter":"ADMIN","dataRangeEnd":"1640995200000","targetModel":"f4f3e9af-6ec8-7806-6a2a-ee616486986b","dataRangeStart":"1638316800000","project":"PM01","className":"org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob","segmentName":"20211201000000_20220101000000","parentId":"05799056-b559-4f8c-8d7c-5db030229392","jobId":"05799056-b559-4f8c-8d7c-5db030229392","outputMetaUrl":"kylin_metadata@jdbc,url=jdbc:mysql://10.30.0.9:3306/kylin,username=kylin,password=******,maxActive=10,maxIdle=10","segmentId":"deae29b0-9702-95f5-7611-467a3d45d015","cuboidsNum":"1023","cubeName":"PM01_M01_C01","jobType":"BUILD","cubeId":"88a4064a-97ce-f002-bf91-6ff2e0d698c1","segmentIds":"deae29b0-9702-95f5-7611-467a3d45d015"}
2022-01-06 11:29:06,074 INFO  [pool-3-thread-1] utils.MetaDumpUtil : Ready to load KylinConfig from uri: kylin_metadata@hdfs,path=hdfs://master:8020/kylin/kylin_metadata/PM01/job_tmp/05799056-b559-4f8c-8d7c-5db030229392-00/meta
2022-01-06 11:29:06,125 INFO  [pool-3-thread-1] common.KylinConfigBase : Kylin Config was updated with kylin.metadata.url.identifier : kylin_metadata
2022-01-06 11:29:06,126 INFO  [pool-3-thread-1] common.KylinConfigBase : Kylin Config was updated with kylin.log.spark-executor-properties-file : /opt/kylin/conf/spark-executor-log4j.properties
2022-01-06 11:29:06,126 INFO  [pool-3-thread-1] common.KylinConfigBase : Kylin Config was updated with kylin.source.provider.0 : org.apache.kylin.engine.spark.source.HiveSource
2022-01-06 11:29:06,146 INFO  [pool-3-thread-1] util.TimeZoneUtils : System timezone set to Asia/Shanghai, TimeZoneId: Asia/Shanghai.
2022-01-06 11:29:06,166 ERROR [pool-3-thread-1] application.SparkApplication : The spark job execute failed!
java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found.
	at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:870)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:290)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:94)
	at org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2022-01-06 11:29:06,168 ERROR [spark-entry-event-loop] application.JobMonitor : Job failed the 0 times.
java.lang.RuntimeException: Error execute org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:97)
	at org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found.
	at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:870)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:290)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:94)
	... 4 more
2022-01-06 11:29:06,175 ERROR [spark-entry-event-loop] application.JobWorkSpace : Job failed eventually. Reason: Error occurred when generate retry configuration.
java.util.NoSuchElementException: spark.executor.memory
	at org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:246)
	at org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:246)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.SparkConf.get(SparkConf.scala:246)
	at org.apache.spark.autoheal.ExceptionTerminator$.incMemory(ExceptionTerminator.scala:70)
	at org.apache.spark.autoheal.ExceptionTerminator$.resolveException(ExceptionTerminator.scala:45)
	at org.apache.spark.application.JobMonitor.handleResourceLack(JobMonitor.scala:53)
	at org.apache.spark.application.JobMonitor$$anon$1.onReceive(JobMonitor.scala:33)
	at org.apache.spark.scheduler.KylinJobEventLoop$$anonfun$onReceive$1.apply(KylinJobEventLoop.scala:42)
	at org.apache.spark.scheduler.KylinJobEventLoop$$anonfun$onReceive$1.apply(KylinJobEventLoop.scala:42)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
	at org.apache.spark.scheduler.KylinJobEventLoop.onReceive(KylinJobEventLoop.scala:42)
	at org.apache.spark.scheduler.KylinJobEventLoop.onReceive(KylinJobEventLoop.scala:29)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
2022-01-06 11:29:06,178 INFO  [spark-entry-event-loop] application.SparkApplication : 
==========================[RESOURCE DETECT BEFORE CUBE]===============================
spark plans :
  
==========================[RESOURCE DETECT BEFORE CUBE]===============================
     
2022-01-06 14:09:38,898 WARN  [pool-3-thread-1] util.NativeCodeLoader : Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2022-01-06 14:09:39,164 INFO  [pool-3-thread-1] application.SparkApplication : Executor task org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob with args : {"distMetaUrl":"kylin_metadata@hdfs,path=hdfs://master:8020/kylin/kylin_metadata/PM01/job_tmp/05799056-b559-4f8c-8d7c-5db030229392-00/meta","submitter":"ADMIN","dataRangeEnd":"1640995200000","targetModel":"f4f3e9af-6ec8-7806-6a2a-ee616486986b","dataRangeStart":"1638316800000","project":"PM01","className":"org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob","segmentName":"20211201000000_20220101000000","parentId":"05799056-b559-4f8c-8d7c-5db030229392","jobId":"05799056-b559-4f8c-8d7c-5db030229392","outputMetaUrl":"kylin_metadata@jdbc,url=jdbc:mysql://10.30.0.9:3306/kylin,username=kylin,password=******,maxActive=10,maxIdle=10","segmentId":"deae29b0-9702-95f5-7611-467a3d45d015","cuboidsNum":"1023","cubeName":"PM01_M01_C01","jobType":"BUILD","cubeId":"88a4064a-97ce-f002-bf91-6ff2e0d698c1","segmentIds":"deae29b0-9702-95f5-7611-467a3d45d015"}
2022-01-06 14:09:39,167 INFO  [pool-3-thread-1] utils.MetaDumpUtil : Ready to load KylinConfig from uri: kylin_metadata@hdfs,path=hdfs://master:8020/kylin/kylin_metadata/PM01/job_tmp/05799056-b559-4f8c-8d7c-5db030229392-00/meta
2022-01-06 14:09:39,218 INFO  [pool-3-thread-1] common.KylinConfigBase : Kylin Config was updated with kylin.metadata.url.identifier : kylin_metadata
2022-01-06 14:09:39,219 INFO  [pool-3-thread-1] common.KylinConfigBase : Kylin Config was updated with kylin.log.spark-executor-properties-file : /opt/kylin/conf/spark-executor-log4j.properties
2022-01-06 14:09:39,219 INFO  [pool-3-thread-1] common.KylinConfigBase : Kylin Config was updated with kylin.source.provider.0 : org.apache.kylin.engine.spark.source.HiveSource
2022-01-06 14:09:39,240 INFO  [pool-3-thread-1] util.TimeZoneUtils : System timezone set to Asia/Shanghai, TimeZoneId: Asia/Shanghai.
2022-01-06 14:09:39,260 ERROR [pool-3-thread-1] application.SparkApplication : The spark job execute failed!
java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found.
	at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:870)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:290)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:94)
	at org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2022-01-06 14:09:39,262 ERROR [spark-entry-event-loop] application.JobMonitor : Job failed the 0 times.
java.lang.RuntimeException: Error execute org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:97)
	at org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found.
	at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:870)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:290)
	at org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:94)
	... 4 more
2022-01-06 14:09:39,270 ERROR [spark-entry-event-loop] application.JobWorkSpace : Job failed eventually. Reason: Error occurred when generate retry configuration.
java.util.NoSuchElementException: spark.executor.memory
	at org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:246)
	at org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:246)
	at scala.Option.getOrElse(Option.scala:121)
	at org.apache.spark.SparkConf.get(SparkConf.scala:246)
	at org.apache.spark.autoheal.ExceptionTerminator$.incMemory(ExceptionTerminator.scala:70)
	at org.apache.spark.autoheal.ExceptionTerminator$.resolveException(ExceptionTerminator.scala:45)
	at org.apache.spark.application.JobMonitor.handleResourceLack(JobMonitor.scala:53)
	at org.apache.spark.application.JobMonitor$$anon$1.onReceive(JobMonitor.scala:33)
	at org.apache.spark.scheduler.KylinJobEventLoop$$anonfun$onReceive$1.apply(KylinJobEventLoop.scala:42)
	at org.apache.spark.scheduler.KylinJobEventLoop$$anonfun$onReceive$1.apply(KylinJobEventLoop.scala:42)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
	at org.apache.spark.scheduler.KylinJobEventLoop.onReceive(KylinJobEventLoop.scala:42)
	at org.apache.spark.scheduler.KylinJobEventLoop.onReceive(KylinJobEventLoop.scala:29)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49)
2022-01-06 14:09:39,273 INFO  [spark-entry-event-loop] application.SparkApplication : 
==========================[RESOURCE DETECT BEFORE CUBE]===============================
spark plans :
  
==========================[RESOURCE DETECT BEFORE CUBE]===============================