org.apache.kylin.engine.spark.exception.SparkException: OS command error exit with return code: 1, error message: 2019-10-15 03:38:26 WARN  SparkConf:66 - The configuration key 'spark.yarn.executor.memoryOverhead' has been deprecated as of Spark 2.3 and may be removed in the future. Please use the new key 'spark.executor.memoryOverhead' instead.
2019-10-15 03:38:26 WARN  SparkConf:66 - The configuration key 'spark.yarn.driver.memoryOverhead' has been deprecated as of Spark 2.3 and may be removed in the future. Please use the new key 'spark.driver.memoryOverhead' instead.
SparkEntry args:-className org.apache.kylin.engine.spark.SparkFactDistinct -counterOutput hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/counter -statisticssamplingpercent 100 -cubename kylin_sales_cube_spark -hiveTable default.kylin_intermediate_kylin_sales_cube_spark_fd5f71d1_c2eb_2af8_59d3_3de193c591b4 -output hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/fact_distinct_columns -input hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_intermediate_kylin_sales_cube_spark_fd5f71d1_c2eb_2af8_59d3_3de193c591b4 -segmentId fd5f71d1-c2eb-2af8-59d3-3de193c591b4 -metaUrl kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
Running org.apache.kylin.engine.spark.SparkFactDistinct -counterOutput hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/counter -statisticssamplingpercent 100 -cubename kylin_sales_cube_spark -hiveTable default.kylin_intermediate_kylin_sales_cube_spark_fd5f71d1_c2eb_2af8_59d3_3de193c591b4 -output hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/fact_distinct_columns -input hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_intermediate_kylin_sales_cube_spark_fd5f71d1_c2eb_2af8_59d3_3de193c591b4 -segmentId fd5f71d1-c2eb-2af8-59d3-3de193c591b4 -metaUrl kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
2019-10-15 03:38:27 WARN  SparkConf:66 - The configuration key 'spark.yarn.executor.memoryOverhead' has been deprecated as of Spark 2.3 and may be removed in the future. Please use the new key 'spark.executor.memoryOverhead' instead.
2019-10-15 03:38:27 WARN  SparkConf:66 - The configuration key 'spark.yarn.driver.memoryOverhead' has been deprecated as of Spark 2.3 and may be removed in the future. Please use the new key 'spark.driver.memoryOverhead' instead.
2019-10-15 03:38:27 INFO  SparkContext:54 - Running Spark version 2.3.2
2019-10-15 03:38:27 INFO  SparkContext:54 - Submitted application: Fact distinct columns for:kylin_sales_cube_spark segment fd5f71d1-c2eb-2af8-59d3-3de193c591b4
2019-10-15 03:38:27 INFO  SecurityManager:54 - Changing view acls to: root
2019-10-15 03:38:27 INFO  SecurityManager:54 - Changing modify acls to: root
2019-10-15 03:38:27 INFO  SecurityManager:54 - Changing view acls groups to: 
2019-10-15 03:38:27 INFO  SecurityManager:54 - Changing modify acls groups to: 
2019-10-15 03:38:27 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2019-10-15 03:38:27 INFO  Utils:54 - Successfully started service 'sparkDriver' on port 42873.
2019-10-15 03:38:27 INFO  SparkEnv:54 - Registering MapOutputTracker
2019-10-15 03:38:27 INFO  SparkEnv:54 - Registering BlockManagerMaster
2019-10-15 03:38:27 INFO  BlockManagerMasterEndpoint:54 - Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
2019-10-15 03:38:27 INFO  BlockManagerMasterEndpoint:54 - BlockManagerMasterEndpoint up
2019-10-15 03:38:27 INFO  DiskBlockManager:54 - Created local directory at /tmp/blockmgr-0af1e005-ecf8-4694-9534-a82a3b120ce9
2019-10-15 03:38:27 INFO  MemoryStore:54 - MemoryStore started with capacity 912.3 MB
2019-10-15 03:38:28 INFO  SparkEnv:54 - Registering OutputCommitCoordinator
2019-10-15 03:38:28 INFO  log:192 - Logging initialized @2669ms
2019-10-15 03:38:28 INFO  Server:351 - jetty-9.3.z-SNAPSHOT, build timestamp: unknown, git hash: unknown
2019-10-15 03:38:28 INFO  Server:419 - Started @2777ms
2019-10-15 03:38:28 INFO  AbstractConnector:278 - Started ServerConnector@2160e52a{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2019-10-15 03:38:28 INFO  Utils:54 - Successfully started service 'SparkUI' on port 4040.
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@534ca02b{/jobs,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5f78de22{/jobs/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@516ebdf8{/jobs/job,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@3eba57a7{/jobs/job/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@272a179c{/stages,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@30feffc{/stages/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@67207d8a{/stages/stage,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@a619c2{/stages/stage/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@648ee871{/stages/pool,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@375b5b7f{/stages/pool/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@1813f3e9{/storage,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@28cb9120{/storage/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@3b152928{/storage/rdd,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@56781d96{/storage/rdd/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5173200b{/environment,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@25c5e994{/environment/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@378bd86d{/executors,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@2189e7a7{/executors/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@69b2f8e5{/executors/threadDump,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@6331250e{/executors/threadDump/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@a10c1b5{/static,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@63429932{/,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@69ce2f62{/api,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@d02f8d{/jobs/job/kill,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@60baef24{/stages/stage/kill,null,AVAILABLE,@Spark}
2019-10-15 03:38:28 INFO  SparkUI:54 - Bound SparkUI to 0.0.0.0, and started at http://sandbox.hortonworks.com:4040
2019-10-15 03:38:28 INFO  SparkContext:54 - Added JAR file:/root/wangrupeng/kylin-spark/KYLIN-4187/lib/kylin-job-3.0.0-SNAPSHOT.jar at spark://sandbox.hortonworks.com:42873/jars/kylin-job-3.0.0-SNAPSHOT.jar with timestamp 1571110708480
2019-10-15 03:38:29 INFO  RMProxy:98 - Connecting to ResourceManager at sandbox.hortonworks.com/10.1.2.13:8050
2019-10-15 03:38:29 INFO  Client:54 - Requesting a new application from cluster with 1 NodeManagers
2019-10-15 03:38:29 INFO  Client:54 - Verifying our application has not requested more than the maximum memory capability of the cluster (12032 MB per container)
2019-10-15 03:38:29 INFO  Client:54 - Will allocate AM container, with 896 MB memory including 384 MB overhead
2019-10-15 03:38:29 INFO  Client:54 - Setting up container launch context for our AM
2019-10-15 03:38:29 INFO  Client:54 - Setting up the launch environment for our AM container
2019-10-15 03:38:29 INFO  Client:54 - Preparing resources for our AM container
2019-10-15 03:38:30 WARN  Client:66 - Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
2019-10-15 03:38:32 INFO  Client:54 - Uploading resource file:/tmp/spark-b7edd592-e647-41ce-a81a-e893c0b4ea74/__spark_libs__3854696117001448561.zip -> hdfs://sandbox.hortonworks.com:8020/user/root/.sparkStaging/application_1570851216198_1307/__spark_libs__3854696117001448561.zip
2019-10-15 03:38:33 INFO  Client:54 - Uploading resource file:/root/wangrupeng/kylin-spark/KYLIN-4187/lib/kylin-job-3.0.0-SNAPSHOT.jar -> hdfs://sandbox.hortonworks.com:8020/user/root/.sparkStaging/application_1570851216198_1307/kylin-job-3.0.0-SNAPSHOT.jar
2019-10-15 03:38:34 INFO  Client:54 - Uploading resource file:/tmp/spark-b7edd592-e647-41ce-a81a-e893c0b4ea74/__spark_conf__2200872594234892213.zip -> hdfs://sandbox.hortonworks.com:8020/user/root/.sparkStaging/application_1570851216198_1307/__spark_conf__.zip
2019-10-15 03:38:34 INFO  SecurityManager:54 - Changing view acls to: root
2019-10-15 03:38:34 INFO  SecurityManager:54 - Changing modify acls to: root
2019-10-15 03:38:34 INFO  SecurityManager:54 - Changing view acls groups to: 
2019-10-15 03:38:34 INFO  SecurityManager:54 - Changing modify acls groups to: 
2019-10-15 03:38:34 INFO  SecurityManager:54 - SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
2019-10-15 03:38:34 INFO  Client:54 - Submitting application application_1570851216198_1307 to ResourceManager
2019-10-15 03:38:34 INFO  YarnClientImpl:273 - Submitted application application_1570851216198_1307
2019-10-15 03:38:34 INFO  SchedulerExtensionServices:54 - Starting Yarn extension services with app application_1570851216198_1307 and attemptId None
2019-10-15 03:38:35 INFO  Client:54 - Application report for application_1570851216198_1307 (state: ACCEPTED)
2019-10-15 03:38:35 INFO  Client:54 - 
	 client token: N/A
	 diagnostics: N/A
	 ApplicationMaster host: N/A
	 ApplicationMaster RPC port: -1
	 queue: default
	 start time: 1571110714197
	 final status: UNDEFINED
	 tracking URL: http://sandbox.hortonworks.com:8088/proxy/application_1570851216198_1307/
	 user: root
2019-10-15 03:38:36 INFO  Client:54 - Application report for application_1570851216198_1307 (state: ACCEPTED)
2019-10-15 03:38:37 INFO  Client:54 - Application report for application_1570851216198_1307 (state: ACCEPTED)
2019-10-15 03:38:38 INFO  Client:54 - Application report for application_1570851216198_1307 (state: ACCEPTED)
2019-10-15 03:38:39 INFO  Client:54 - Application report for application_1570851216198_1307 (state: ACCEPTED)
2019-10-15 03:38:40 INFO  Client:54 - Application report for application_1570851216198_1307 (state: ACCEPTED)
2019-10-15 03:38:40 INFO  YarnClientSchedulerBackend:54 - Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> sandbox.hortonworks.com, PROXY_URI_BASES -> http://sandbox.hortonworks.com:8088/proxy/application_1570851216198_1307), /proxy/application_1570851216198_1307
2019-10-15 03:38:40 INFO  JettyUtils:54 - Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /jobs, /jobs/json, /jobs/job, /jobs/job/json, /stages, /stages/json, /stages/stage, /stages/stage/json, /stages/pool, /stages/pool/json, /storage, /storage/json, /storage/rdd, /storage/rdd/json, /environment, /environment/json, /executors, /executors/json, /executors/threadDump, /executors/threadDump/json, /static, /, /api, /jobs/job/kill, /stages/stage/kill.
2019-10-15 03:38:40 INFO  YarnSchedulerBackend$YarnSchedulerEndpoint:54 - ApplicationMaster registered as NettyRpcEndpointRef(spark-client://YarnAM)
2019-10-15 03:38:41 INFO  Client:54 - Application report for application_1570851216198_1307 (state: RUNNING)
2019-10-15 03:38:41 INFO  Client:54 - 
	 client token: N/A
	 diagnostics: N/A
	 ApplicationMaster host: 10.1.2.13
	 ApplicationMaster RPC port: 0
	 queue: default
	 start time: 1571110714197
	 final status: UNDEFINED
	 tracking URL: http://sandbox.hortonworks.com:8088/proxy/application_1570851216198_1307/
	 user: root
2019-10-15 03:38:41 INFO  YarnClientSchedulerBackend:54 - Application application_1570851216198_1307 has started running.
2019-10-15 03:38:41 INFO  Utils:54 - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 53367.
2019-10-15 03:38:41 INFO  NettyBlockTransferService:54 - Server created on sandbox.hortonworks.com:53367
2019-10-15 03:38:41 INFO  BlockManager:54 - Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
2019-10-15 03:38:41 INFO  BlockManagerMaster:54 - Registering BlockManager BlockManagerId(driver, sandbox.hortonworks.com, 53367, None)
2019-10-15 03:38:41 INFO  BlockManagerMasterEndpoint:54 - Registering block manager sandbox.hortonworks.com:53367 with 912.3 MB RAM, BlockManagerId(driver, sandbox.hortonworks.com, 53367, None)
2019-10-15 03:38:41 INFO  BlockManagerMaster:54 - Registered BlockManager BlockManagerId(driver, sandbox.hortonworks.com, 53367, None)
2019-10-15 03:38:41 INFO  BlockManager:54 - external shuffle service port = 7337
2019-10-15 03:38:41 INFO  BlockManager:54 - Initialized BlockManager: BlockManagerId(driver, sandbox.hortonworks.com, 53367, None)
2019-10-15 03:38:41 INFO  JettyUtils:54 - Adding filter org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter to /metrics/json.
2019-10-15 03:38:41 INFO  ContextHandler:781 - Started o.s.j.s.ServletContextHandler@5c8d58ed{/metrics/json,null,AVAILABLE,@Spark}
2019-10-15 03:38:41 INFO  EventLoggingListener:54 - Logging events to hdfs:/kylin/spark-history/application_1570851216198_1307
2019-10-15 03:38:44 INFO  YarnSchedulerBackend$YarnDriverEndpoint:54 - Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.1.2.13:47409) with ID 1
2019-10-15 03:38:44 INFO  YarnClientSchedulerBackend:54 - SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
2019-10-15 03:38:44 INFO  BlockManagerMasterEndpoint:54 - Registering block manager sandbox.hortonworks.com:60859 with 229.8 MB RAM, BlockManagerId(1, sandbox.hortonworks.com, 60859, None)
2019-10-15 03:38:44 INFO  AbstractHadoopJob:518 - Ready to load KylinConfig from uri: kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
2019-10-15 03:38:44 INFO  KylinConfig:461 - Creating new manager instance of class org.apache.kylin.cube.CubeManager
2019-10-15 03:38:44 INFO  CubeManager:134 - Initializing CubeManager with config kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
2019-10-15 03:38:44 INFO  ResourceStore:91 - Using metadata url kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata for resource store
2019-10-15 03:38:44 INFO  HDFSResourceStore:74 - hdfs meta path : hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
2019-10-15 03:38:45 INFO  KylinConfig:461 - Creating new manager instance of class org.apache.kylin.cube.CubeDescManager
2019-10-15 03:38:45 INFO  CubeDescManager:91 - Initializing CubeDescManager with config kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
2019-10-15 03:38:45 INFO  KylinConfig:461 - Creating new manager instance of class org.apache.kylin.metadata.project.ProjectManager
2019-10-15 03:38:45 INFO  ProjectManager:81 - Initializing ProjectManager with metadata url kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
2019-10-15 03:38:45 INFO  KylinConfig:461 - Creating new manager instance of class org.apache.kylin.metadata.cachesync.Broadcaster
2019-10-15 03:38:45 INFO  KylinConfig:461 - Creating new manager instance of class org.apache.kylin.metadata.model.DataModelManager
2019-10-15 03:38:45 INFO  KylinConfig:461 - Creating new manager instance of class org.apache.kylin.metadata.TableMetadataManager
2019-10-15 03:38:45 INFO  MeasureTypeFactory:117 - Checking custom measure types from kylin config
2019-10-15 03:38:45 INFO  MeasureTypeFactory:146 - registering COUNT_DISTINCT(hllc), class org.apache.kylin.measure.hllc.HLLCMeasureType$Factory
2019-10-15 03:38:45 INFO  MeasureTypeFactory:146 - registering COUNT_DISTINCT(bitmap), class org.apache.kylin.measure.bitmap.BitmapMeasureType$Factory
2019-10-15 03:38:45 INFO  MeasureTypeFactory:146 - registering TOP_N(topn), class org.apache.kylin.measure.topn.TopNMeasureType$Factory
2019-10-15 03:38:45 INFO  MeasureTypeFactory:146 - registering RAW(raw), class org.apache.kylin.measure.raw.RawMeasureType$Factory
2019-10-15 03:38:45 INFO  MeasureTypeFactory:146 - registering EXTENDED_COLUMN(extendedcolumn), class org.apache.kylin.measure.extendedcolumn.ExtendedColumnMeasureType$Factory
2019-10-15 03:38:45 INFO  MeasureTypeFactory:146 - registering PERCENTILE_APPROX(percentile), class org.apache.kylin.measure.percentile.PercentileMeasureType$Factory
2019-10-15 03:38:45 INFO  MeasureTypeFactory:146 - registering COUNT_DISTINCT(dim_dc), class org.apache.kylin.measure.dim.DimCountDistinctMeasureType$Factory
2019-10-15 03:38:45 INFO  DataModelManager:189 - Model kylin_sales_model is missing or unloaded yet
2019-10-15 03:38:45 INFO  DataModelManager:189 - Model kylin_streaming_model is missing or unloaded yet
2019-10-15 03:38:45 INFO  DataModelManager:189 - Model kylin_sales_model_spark is missing or unloaded yet
2019-10-15 03:38:45 INFO  DataModelManager:189 - Model kylin_sales_model_spark_uhc is missing or unloaded yet
2019-10-15 03:38:45 INFO  SparkFactDistinct:182 - RDD Output path: hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/fact_distinct_columns
2019-10-15 03:38:45 INFO  SparkFactDistinct:183 - getTotalReducerNum: 5
2019-10-15 03:38:45 INFO  SparkFactDistinct:184 - getCuboidRowCounterReducerNum: 1
2019-10-15 03:38:45 INFO  SparkFactDistinct:185 - counter path hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/counter
2019-10-15 03:38:45 INFO  ZlibFactory:49 - Successfully loaded & initialized native-zlib library
2019-10-15 03:38:45 INFO  CodecPool:181 - Got brand-new decompressor [.deflate]
2019-10-15 03:38:45 INFO  CodecPool:181 - Got brand-new decompressor [.deflate]
2019-10-15 03:38:45 INFO  CodecPool:181 - Got brand-new decompressor [.deflate]
2019-10-15 03:38:45 INFO  CodecPool:181 - Got brand-new decompressor [.deflate]
2019-10-15 03:38:46 INFO  MemoryStore:54 - Block broadcast_0 stored as values in memory (estimated size 306.4 KB, free 912.0 MB)
2019-10-15 03:38:46 INFO  MemoryStore:54 - Block broadcast_0_piece0 stored as bytes in memory (estimated size 27.1 KB, free 912.0 MB)
2019-10-15 03:38:46 INFO  BlockManagerInfo:54 - Added broadcast_0_piece0 in memory on sandbox.hortonworks.com:53367 (size: 27.1 KB, free: 912.3 MB)
2019-10-15 03:38:46 INFO  SparkContext:54 - Created broadcast 0 from sequenceFile at SparkUtil.java:159
2019-10-15 03:38:46 INFO  FileOutputCommitter:108 - File Output Committer Algorithm version is 1
2019-10-15 03:38:47 INFO  SparkContext:54 - Starting job: runJob at MultipleOutputsRDD.scala:97
2019-10-15 03:38:47 INFO  FileInputFormat:249 - Total input paths to process : 1
2019-10-15 03:38:47 INFO  DAGScheduler:54 - Registering RDD 3 (mapPartitionsToPair at SparkFactDistinct.java:200)
2019-10-15 03:38:47 INFO  DAGScheduler:54 - Got job 0 (runJob at MultipleOutputsRDD.scala:97) with 5 output partitions
2019-10-15 03:38:47 INFO  DAGScheduler:54 - Final stage: ResultStage 1 (runJob at MultipleOutputsRDD.scala:97)
2019-10-15 03:38:47 INFO  DAGScheduler:54 - Parents of final stage: List(ShuffleMapStage 0)
2019-10-15 03:38:47 INFO  DAGScheduler:54 - Missing parents: List(ShuffleMapStage 0)
2019-10-15 03:38:47 INFO  DAGScheduler:54 - Submitting ShuffleMapStage 0 (MapPartitionsRDD[3] at mapPartitionsToPair at SparkFactDistinct.java:200), which has no missing parents
2019-10-15 03:38:47 INFO  MemoryStore:54 - Block broadcast_1 stored as values in memory (estimated size 80.9 KB, free 911.9 MB)
2019-10-15 03:38:47 INFO  MemoryStore:54 - Block broadcast_1_piece0 stored as bytes in memory (estimated size 31.3 KB, free 911.9 MB)
2019-10-15 03:38:47 INFO  BlockManagerInfo:54 - Added broadcast_1_piece0 in memory on sandbox.hortonworks.com:53367 (size: 31.3 KB, free: 912.2 MB)
2019-10-15 03:38:47 INFO  SparkContext:54 - Created broadcast 1 from broadcast at DAGScheduler.scala:1039
2019-10-15 03:38:47 INFO  DAGScheduler:54 - Submitting 2 missing tasks from ShuffleMapStage 0 (MapPartitionsRDD[3] at mapPartitionsToPair at SparkFactDistinct.java:200) (first 15 tasks are for partitions Vector(0, 1))
2019-10-15 03:38:47 INFO  YarnScheduler:54 - Adding task set 0.0 with 2 tasks
2019-10-15 03:38:47 INFO  TaskSetManager:54 - Starting task 0.0 in stage 0.0 (TID 0, sandbox.hortonworks.com, executor 1, partition 0, NODE_LOCAL, 8048 bytes)
2019-10-15 03:38:47 INFO  BlockManagerInfo:54 - Added broadcast_1_piece0 in memory on sandbox.hortonworks.com:60859 (size: 31.3 KB, free: 229.8 MB)
2019-10-15 03:38:48 INFO  BlockManagerInfo:54 - Added broadcast_0_piece0 in memory on sandbox.hortonworks.com:60859 (size: 27.1 KB, free: 229.7 MB)
2019-10-15 03:38:50 INFO  TaskSetManager:54 - Starting task 1.0 in stage 0.0 (TID 1, sandbox.hortonworks.com, executor 1, partition 1, NODE_LOCAL, 8048 bytes)
2019-10-15 03:38:50 INFO  TaskSetManager:54 - Finished task 0.0 in stage 0.0 (TID 0) in 3475 ms on sandbox.hortonworks.com (executor 1) (1/2)
2019-10-15 03:38:50 INFO  TaskSetManager:54 - Finished task 1.0 in stage 0.0 (TID 1) in 206 ms on sandbox.hortonworks.com (executor 1) (2/2)
2019-10-15 03:38:50 INFO  DAGScheduler:54 - ShuffleMapStage 0 (mapPartitionsToPair at SparkFactDistinct.java:200) finished in 3.748 s
2019-10-15 03:38:50 INFO  DAGScheduler:54 - looking for newly runnable stages
2019-10-15 03:38:50 INFO  DAGScheduler:54 - running: Set()
2019-10-15 03:38:50 INFO  DAGScheduler:54 - waiting: Set(ResultStage 1)
2019-10-15 03:38:50 INFO  DAGScheduler:54 - failed: Set()
2019-10-15 03:38:50 INFO  YarnScheduler:54 - Removed TaskSet 0.0, whose tasks have all completed, from pool 
2019-10-15 03:38:50 INFO  DAGScheduler:54 - Submitting ResultStage 1 (MapPartitionsRDD[5] at mapPartitionsToPair at SparkFactDistinct.java:213), which has no missing parents
2019-10-15 03:38:50 INFO  MemoryStore:54 - Block broadcast_2 stored as values in memory (estimated size 155.8 KB, free 911.7 MB)
2019-10-15 03:38:50 INFO  MemoryStore:54 - Block broadcast_2_piece0 stored as bytes in memory (estimated size 57.7 KB, free 911.7 MB)
2019-10-15 03:38:50 INFO  BlockManagerInfo:54 - Added broadcast_2_piece0 in memory on sandbox.hortonworks.com:53367 (size: 57.7 KB, free: 912.2 MB)
2019-10-15 03:38:50 INFO  SparkContext:54 - Created broadcast 2 from broadcast at DAGScheduler.scala:1039
2019-10-15 03:38:50 INFO  DAGScheduler:54 - Submitting 5 missing tasks from ResultStage 1 (MapPartitionsRDD[5] at mapPartitionsToPair at SparkFactDistinct.java:213) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4))
2019-10-15 03:38:50 INFO  YarnScheduler:54 - Adding task set 1.0 with 5 tasks
2019-10-15 03:38:50 INFO  TaskSetManager:54 - Starting task 0.0 in stage 1.0 (TID 2, sandbox.hortonworks.com, executor 1, partition 0, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:50 INFO  BlockManagerInfo:54 - Added broadcast_2_piece0 in memory on sandbox.hortonworks.com:60859 (size: 57.7 KB, free: 229.7 MB)
2019-10-15 03:38:51 INFO  MapOutputTrackerMasterEndpoint:54 - Asked to send map output locations for shuffle 0 to 10.1.2.13:47409
2019-10-15 03:38:51 INFO  TaskSetManager:54 - Starting task 1.0 in stage 1.0 (TID 3, sandbox.hortonworks.com, executor 1, partition 1, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:51 WARN  TaskSetManager:66 - Lost task 0.0 in stage 1.0 (TID 2, sandbox.hortonworks.com, executor 1): org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them
	at org.apache.kylin.common.KylinConfig.getSitePropertiesFile(KylinConfig.java:336)
	at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
	at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:358)
	at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:137)
	at org.apache.kylin.dict.CacheDictionary.enableCache(CacheDictionary.java:105)
	at org.apache.kylin.dict.TrieDictionary.init(TrieDictionary.java:119)
	at org.apache.kylin.dict.TrieDictionary.<init>(TrieDictionary.java:89)
	at org.apache.kylin.dict.TrieDictionaryBuilder.build(TrieDictionaryBuilder.java:419)
	at org.apache.kylin.dict.TrieDictionaryForestBuilder.build(TrieDictionaryForestBuilder.java:110)
	at org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.build(DictionaryGenerator.java:312)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:774)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:650)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

2019-10-15 03:38:51 INFO  TaskSetManager:54 - Starting task 0.1 in stage 1.0 (TID 4, sandbox.hortonworks.com, executor 1, partition 0, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:51 INFO  TaskSetManager:54 - Finished task 1.0 in stage 1.0 (TID 3) in 386 ms on sandbox.hortonworks.com (executor 1) (1/5)
2019-10-15 03:38:51 INFO  TaskSetManager:54 - Starting task 2.0 in stage 1.0 (TID 5, sandbox.hortonworks.com, executor 1, partition 2, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:51 INFO  TaskSetManager:54 - Lost task 0.1 in stage 1.0 (TID 4) on sandbox.hortonworks.com, executor 1: org.apache.kylin.common.KylinConfigCannotInitException (Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them) [duplicate 1]
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Starting task 0.2 in stage 1.0 (TID 6, sandbox.hortonworks.com, executor 1, partition 0, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Lost task 2.0 in stage 1.0 (TID 5) on sandbox.hortonworks.com, executor 1: org.apache.kylin.common.KylinConfigCannotInitException (Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them) [duplicate 2]
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Starting task 2.1 in stage 1.0 (TID 7, sandbox.hortonworks.com, executor 1, partition 2, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Lost task 0.2 in stage 1.0 (TID 6) on sandbox.hortonworks.com, executor 1: org.apache.kylin.common.KylinConfigCannotInitException (Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them) [duplicate 3]
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Starting task 0.3 in stage 1.0 (TID 8, sandbox.hortonworks.com, executor 1, partition 0, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Lost task 2.1 in stage 1.0 (TID 7) on sandbox.hortonworks.com, executor 1: org.apache.kylin.common.KylinConfigCannotInitException (Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them) [duplicate 4]
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Starting task 2.2 in stage 1.0 (TID 9, sandbox.hortonworks.com, executor 1, partition 2, NODE_LOCAL, 7660 bytes)
2019-10-15 03:38:52 INFO  TaskSetManager:54 - Lost task 0.3 in stage 1.0 (TID 8) on sandbox.hortonworks.com, executor 1: org.apache.kylin.common.KylinConfigCannotInitException (Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them) [duplicate 5]
2019-10-15 03:38:52 ERROR TaskSetManager:70 - Task 0 in stage 1.0 failed 4 times; aborting job
2019-10-15 03:38:52 INFO  YarnScheduler:54 - Cancelling stage 1
2019-10-15 03:38:52 INFO  YarnScheduler:54 - Stage 1 was cancelled
2019-10-15 03:38:52 INFO  DAGScheduler:54 - ResultStage 1 (runJob at MultipleOutputsRDD.scala:97) failed in 1.609 s due to Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 8, sandbox.hortonworks.com, executor 1): org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them
	at org.apache.kylin.common.KylinConfig.getSitePropertiesFile(KylinConfig.java:336)
	at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
	at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:358)
	at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:137)
	at org.apache.kylin.dict.CacheDictionary.enableCache(CacheDictionary.java:105)
	at org.apache.kylin.dict.TrieDictionary.init(TrieDictionary.java:119)
	at org.apache.kylin.dict.TrieDictionary.<init>(TrieDictionary.java:89)
	at org.apache.kylin.dict.TrieDictionaryBuilder.build(TrieDictionaryBuilder.java:419)
	at org.apache.kylin.dict.TrieDictionaryForestBuilder.build(TrieDictionaryForestBuilder.java:110)
	at org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.build(DictionaryGenerator.java:312)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:774)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:650)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
2019-10-15 03:38:52 INFO  DAGScheduler:54 - Job 0 failed: runJob at MultipleOutputsRDD.scala:97, took 5.467997 s
2019-10-15 03:38:52 INFO  AbstractConnector:318 - Stopped Spark@2160e52a{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}
2019-10-15 03:38:52 INFO  SparkUI:54 - Stopped Spark web UI at http://sandbox.hortonworks.com:4040
2019-10-15 03:38:52 INFO  YarnClientSchedulerBackend:54 - Interrupting monitor thread
2019-10-15 03:38:52 INFO  YarnClientSchedulerBackend:54 - Shutting down all executors
2019-10-15 03:38:52 INFO  YarnSchedulerBackend$YarnDriverEndpoint:54 - Asking each executor to shut down
2019-10-15 03:38:52 INFO  SchedulerExtensionServices:54 - Stopping SchedulerExtensionServices
(serviceOption=None,
 services=List(),
 started=false)
2019-10-15 03:38:52 INFO  YarnClientSchedulerBackend:54 - Stopped
2019-10-15 03:38:52 ERROR TransportRequestHandler:210 - Error while invoking RpcHandler#receive() for one-way message.
org.apache.spark.SparkException: Could not find CoarseGrainedScheduler.
	at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:160)
	at org.apache.spark.rpc.netty.Dispatcher.postOneWayMessage(Dispatcher.scala:140)
	at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:655)
	at org.apache.spark.network.server.TransportRequestHandler.processOneWayMessage(TransportRequestHandler.java:208)
	at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:113)
	at org.apache.spark.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:118)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at org.apache.spark.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:85)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1359)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:935)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:138)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
	at java.lang.Thread.run(Thread.java:748)
2019-10-15 03:38:52 INFO  MapOutputTrackerMasterEndpoint:54 - MapOutputTrackerMasterEndpoint stopped!
2019-10-15 03:38:52 INFO  MemoryStore:54 - MemoryStore cleared
2019-10-15 03:38:52 INFO  BlockManager:54 - BlockManager stopped
2019-10-15 03:38:52 INFO  BlockManagerMaster:54 - BlockManagerMaster stopped
2019-10-15 03:38:52 INFO  OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:54 - OutputCommitCoordinator stopped!
2019-10-15 03:38:52 INFO  SparkContext:54 - Successfully stopped SparkContext
Exception in thread "main" java.lang.RuntimeException: error execute org.apache.kylin.engine.spark.SparkFactDistinct. Root cause: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 8, sandbox.hortonworks.com, executor 1): org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them
	at org.apache.kylin.common.KylinConfig.getSitePropertiesFile(KylinConfig.java:336)
	at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
	at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:358)
	at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:137)
	at org.apache.kylin.dict.CacheDictionary.enableCache(CacheDictionary.java:105)
	at org.apache.kylin.dict.TrieDictionary.init(TrieDictionary.java:119)
	at org.apache.kylin.dict.TrieDictionary.<init>(TrieDictionary.java:89)
	at org.apache.kylin.dict.TrieDictionaryBuilder.build(TrieDictionaryBuilder.java:419)
	at org.apache.kylin.dict.TrieDictionaryForestBuilder.build(TrieDictionaryForestBuilder.java:110)
	at org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.build(DictionaryGenerator.java:312)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:774)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:650)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
	at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:42)
	at org.apache.kylin.common.util.SparkEntry.main(SparkEntry.java:44)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:894)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 8, sandbox.hortonworks.com, executor 1): org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them
	at org.apache.kylin.common.KylinConfig.getSitePropertiesFile(KylinConfig.java:336)
	at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
	at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:358)
	at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:137)
	at org.apache.kylin.dict.CacheDictionary.enableCache(CacheDictionary.java:105)
	at org.apache.kylin.dict.TrieDictionary.init(TrieDictionary.java:119)
	at org.apache.kylin.dict.TrieDictionary.<init>(TrieDictionary.java:89)
	at org.apache.kylin.dict.TrieDictionaryBuilder.build(TrieDictionaryBuilder.java:419)
	at org.apache.kylin.dict.TrieDictionaryForestBuilder.build(TrieDictionaryForestBuilder.java:110)
	at org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.build(DictionaryGenerator.java:312)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:774)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:650)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Driver stacktrace:
	at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1651)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1639)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1638)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
	at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1638)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
	at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
	at scala.Option.foreach(Option.scala:257)
	at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1872)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1821)
	at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1810)
	at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
	at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2034)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2055)
	at org.apache.spark.SparkContext.runJob(SparkContext.scala:2087)
	at org.apache.kylin.engine.spark.MultipleOutputsRDD.saveAsNewAPIHadoopDatasetWithMultipleOutputs(MultipleOutputsRDD.scala:97)
	at org.apache.kylin.engine.spark.SparkFactDistinct.execute(SparkFactDistinct.java:233)
	at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37)
	... 11 more
Caused by: org.apache.kylin.common.KylinConfigCannotInitException: Didn't find KYLIN_CONF or KYLIN_HOME, please set one of them
	at org.apache.kylin.common.KylinConfig.getSitePropertiesFile(KylinConfig.java:336)
	at org.apache.kylin.common.KylinConfig.buildSiteOrderedProps(KylinConfig.java:378)
	at org.apache.kylin.common.KylinConfig.buildSiteProperties(KylinConfig.java:358)
	at org.apache.kylin.common.KylinConfig.getInstanceFromEnv(KylinConfig.java:137)
	at org.apache.kylin.dict.CacheDictionary.enableCache(CacheDictionary.java:105)
	at org.apache.kylin.dict.TrieDictionary.init(TrieDictionary.java:119)
	at org.apache.kylin.dict.TrieDictionary.<init>(TrieDictionary.java:89)
	at org.apache.kylin.dict.TrieDictionaryBuilder.build(TrieDictionaryBuilder.java:419)
	at org.apache.kylin.dict.TrieDictionaryForestBuilder.build(TrieDictionaryForestBuilder.java:110)
	at org.apache.kylin.dict.DictionaryGenerator$NumberTrieDictForestBuilder.build(DictionaryGenerator.java:312)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:774)
	at org.apache.kylin.engine.spark.SparkFactDistinct$MultiOutputFunction.call(SparkFactDistinct.java:650)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$7$1.apply(JavaRDDLike.scala:186)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(RDD.scala:801)
	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
	at org.apache.spark.scheduler.Task.run(Task.scala:109)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2019-10-15 03:38:52 INFO  ShutdownHookManager:54 - Shutdown hook called
2019-10-15 03:38:52 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-b7edd592-e647-41ce-a81a-e893c0b4ea74
2019-10-15 03:38:52 INFO  ShutdownHookManager:54 - Deleting directory /tmp/spark-92658ef5-3679-4047-a913-00dfa93a9bbb
The command is: 
export HADOOP_CONF_DIR=/usr/hdp/2.4.0.0-169/hadoop/conf && /root/spark-2.3.2-bin-hadoop2.7/bin/spark-submit --class org.apache.kylin.common.util.SparkEntry --name "Extract Fact Table Distinct Columns:kylin_sales_cube_spark[20120101000000_20120201000000]" --conf spark.executor.instances=1  --conf spark.yarn.queue=default  --conf spark.yarn.am.extraJavaOptions=-Dhdp.version=current  --conf spark.history.fs.logDirectory=hdfs:///kylin/spark-history  --conf spark.driver.extraJavaOptions=-Dhdp.version=current  --conf spark.master=yarn  --conf spark.executor.extraJavaOptions=-Dhdp.version=current  --conf spark.hadoop.yarn.timeline-service.enabled=false  --conf spark.executor.memory=768M  --conf spark.eventLog.enabled=true  --conf spark.eventLog.dir=hdfs:///kylin/spark-history  --conf spark.executor.cores=1  --conf spark.yarn.executor.memoryOverhead=384  --conf spark.yarn.submit.file.replication=1  --conf spark.driver.memory=2G  --conf spark.shuffle.service.enabled=true  --conf spark.yarn.driver.memoryOverhead=256 --jars /root/wangrupeng/kylin-spark/KYLIN-4187/lib/kylin-job-3.0.0-SNAPSHOT.jar /root/wangrupeng/kylin-spark/KYLIN-4187/lib/kylin-job-3.0.0-SNAPSHOT.jar -className org.apache.kylin.engine.spark.SparkFactDistinct -counterOutput hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/counter -statisticssamplingpercent 100 -cubename kylin_sales_cube_spark -hiveTable default.kylin_intermediate_kylin_sales_cube_spark_fd5f71d1_c2eb_2af8_59d3_3de193c591b4 -output hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/fact_distinct_columns -input hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_intermediate_kylin_sales_cube_spark_fd5f71d1_c2eb_2af8_59d3_3de193c591b4 -segmentId fd5f71d1-c2eb-2af8-59d3-3de193c591b4 -metaUrl kylin_metadata_wrp_debug@hdfs,path=hdfs://sandbox.hortonworks.com:8020/kylin/kylin_metadata_wrp_debug/kylin-87b8246a-b4cb-4528-7157-70889d695569/kylin_sales_cube_spark/metadata
	at org.apache.kylin.engine.spark.SparkExecutable.doWork(SparkExecutable.java:392)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179)
	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)