2017-05-22 14:04:30,450 DEBUG [http-bio-7070-exec-7] controller.UserController:64 : authentication.getPrincipal() is org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
2017-05-22 14:04:36,582 INFO  [http-bio-7070-exec-1] cube.CubeManager:366 : Updating cube instance 'cc'
2017-05-22 14:04:36,583 WARN  [http-bio-7070-exec-1] cube.CubeValidator:102 : NEW segment start does not fit/connect with other segments: cc[19700101000000_2922789940817071255]
2017-05-22 14:04:36,583 WARN  [http-bio-7070-exec-1] cube.CubeValidator:104 : NEW segment end does not fit/connect with other segments: cc[19700101000000_2922789940817071255]
2017-05-22 14:04:36,590 INFO  [pool-8-thread-1] cachesync.Broadcaster:123 : Announcing new broadcast event: BroadcastEvent{entity=cube, event=update, cacheKey=cc}
2017-05-22 14:04:36,595 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:189 : Broadcasting metadata change: entity=cube, event=UPDATE, cacheKey=cc, listeners=[org.apache.kylin.cube.CubeManager$CubeSyncListener@6a152bc6, org.apache.kylin.rest.service.CacheService$1@52c1d366]
2017-05-22 14:04:36,597 INFO  [http-bio-7070-exec-3] cube.CubeManager:785 : Reloaded cube cc being CUBE[name=cc] having 1 segments
2017-05-22 14:04:36,597 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:189 : Broadcasting metadata change: entity=project_data, event=UPDATE, cacheKey=Kylin_Hive, listeners=[org.apache.kylin.metadata.MetadataManager$SrcTableSyncListener@3242e30a, org.apache.kylin.metadata.MetadataManager$SrcTableExtSyncListener@59bfd82d, org.apache.kylin.metadata.MetadataManager$DataModelSyncListener@1a3793c7, org.apache.kylin.metadata.MetadataManager$ExtFilterSyncListener@46312904, org.apache.kylin.metadata.project.ProjectManager$ProjectSyncListener@634ec87b, org.apache.kylin.cube.CubeDescManager$CubeDescSyncListener@44170a8f, org.apache.kylin.cube.CubeManager$CubeSyncListener@6a152bc6, org.apache.kylin.rest.service.CacheService$1@52c1d366]
2017-05-22 14:04:36,598 INFO  [http-bio-7070-exec-3] service.CacheService:135 : removeOLAPDataSource is called for project Kylin_Hive
2017-05-22 14:04:36,598 INFO  [http-bio-7070-exec-3] service.CacheService:117 : cleaning cache for project Kylin_Hive (currently remove all entries)
2017-05-22 14:04:36,598 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:219 : Done broadcasting metadata change: entity=project_data, event=UPDATE, cacheKey=Kylin_Hive
2017-05-22 14:04:36,599 DEBUG [http-bio-7070-exec-3] cachesync.Broadcaster:219 : Done broadcasting metadata change: entity=cube, event=UPDATE, cacheKey=cc
2017-05-22 14:04:36,606 INFO  [http-bio-7070-exec-1] mr.BatchCubingJobBuilder2:50 : MR_V2 new job to BUILD segment cc[19700101000000_2922789940817071255]
2017-05-22 14:04:36,692 WARN  [http-bio-7070-exec-1] spark.SparkBatchCubingJobBuilder2:81 : failed to locate jar for class org.cloudera.htrace.HTraceConfiguration, ignore it
2017-05-22 14:04:36,719 DEBUG [http-bio-7070-exec-1] service.AclService:243 : ACL of org.springframework.security.acls.domain.ObjectIdentityImpl[Type: org.apache.kylin.job.JobInstance; Identifier: df682969-46e7-4f05-9394-0cbee8247e54] created successfully.
2017-05-22 14:04:36,725 DEBUG [http-bio-7070-exec-1] service.AclService:317 : ACL of org.springframework.security.acls.domain.ObjectIdentityImpl[Type: org.apache.kylin.job.JobInstance; Identifier: df682969-46e7-4f05-9394-0cbee8247e54] updated successfully.
2017-05-22 14:05:25,706 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:114 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} prepare to schedule
2017-05-22 14:05:25,707 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:117 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} scheduled
2017-05-22 14:05:25,708 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] execution.AbstractExecutable:110 : Executing AbstractExecutable (cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36)
2017-05-22 14:05:25,709 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-316] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:05:25,710 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 0 should running, 1 actual running, 0 stopped, 1 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:05:25,711 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from READY to RUNNING
2017-05-22 14:05:25,712 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] execution.AbstractExecutable:110 : Executing AbstractExecutable (Create Intermediate Flat Hive Table)
2017-05-22 14:05:25,712 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-316] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-00
2017-05-22 14:05:25,714 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-00 from READY to RUNNING
2017-05-22 14:05:25,715 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Create and distribute table, cmd: 
2017-05-22 14:05:25,716 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : hive -e "USE default;
SET dfs.replication=2;
SET hive.exec.compress.output=true;
SET hive.auto.convert.join=true;
SET hive.auto.convert.join.noconditionaltask=true;
SET hive.auto.convert.join.noconditionaltask.size=100000000;
SET mapreduce.job.split.metainfo.maxsize=-1;
SET hive.stats.autogather=true;
SET hive.merge.mapfiles=false;
SET hive.merge.mapredfiles=false;
SET mapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
SET mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;

DROP TABLE IF EXISTS kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d;
CREATE EXTERNAL TABLE IF NOT EXISTS kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d
(
LOG_DATA_170416_IP int
,LOG_DATA_170416_U_DOMAIN int
,LOG_DATA_170416_U_PAGE int
,LOG_DATA_170416_R_DOMAIN int
,LOG_DATA_170416_R_PAGE int
,LOG_DATA_170416_AGENT int
)
STORED AS SEQUENCEFILE
LOCATION 'hdfs:///kylin/kylin_metadata/kylin-df682969-46e7-4f05-9394-0cbee8247e54/kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d';
INSERT OVERWRITE TABLE kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d SELECT
LOG_DATA_170416.IP
,LOG_DATA_170416.U_DOMAIN
,LOG_DATA_170416.U_PAGE
,LOG_DATA_170416.R_DOMAIN
,LOG_DATA_170416.R_PAGE
,LOG_DATA_170416.AGENT
FROM DEFAULT.LOG_DATA_170416 as LOG_DATA_170416 
;

"
2017-05-22 14:05:25,857 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
2017-05-22 14:05:28,974 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
2017-05-22 14:05:29,420 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : SLF4J: Class path contains multiple SLF4J bindings.
2017-05-22 14:05:29,421 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2017-05-22 14:05:29,421 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : SLF4J: Found binding in [jar:file:/opt/cloudera/apache-kylin-2.0.0-bin/spark/lib/spark-assembly-1.6.3-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2017-05-22 14:05:29,421 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2017-05-22 14:05:29,423 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-05-22 14:05:29,738 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 17/05/22 14:05:29 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present.  Continuing without it.
2017-05-22 14:05:30,257 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
2017-05-22 14:05:31,597 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 
2017-05-22 14:05:31,597 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/hive-common-1.1.0-cdh5.9.0.jar!/hive-log4j.properties
2017-05-22 14:05:33,400 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : OK
2017-05-22 14:05:33,401 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Time taken: 0.617 seconds
2017-05-22 14:05:33,462 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : OK
2017-05-22 14:05:33,462 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Time taken: 0.054 seconds
2017-05-22 14:05:33,689 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : OK
2017-05-22 14:05:33,694 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Time taken: 0.231 seconds
2017-05-22 14:05:34,149 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Query ID = root_20170522140505_909616fe-d505-4e51-8f70-56a0ab52f0c4
2017-05-22 14:05:34,149 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Total jobs = 1
2017-05-22 14:05:34,165 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Launching Job 1 out of 1
2017-05-22 14:05:34,172 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Number of reduce tasks is set to 0 since there's no reduce operator
2017-05-22 14:05:36,081 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Starting Job = job_1493346400817_0062, Tracking URL = http://gp-node-master.zoldbs.com.cn:8088/proxy/application_1493346400817_0062/
2017-05-22 14:05:36,082 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Kill Command = /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hadoop/bin/hadoop job  -kill job_1493346400817_0062
2017-05-22 14:05:40,511 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Hadoop job information for Stage-1: number of mappers: 16; number of reducers: 0
2017-05-22 14:05:40,559 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:05:40,555 Stage-1 map = 0%,  reduce = 0%
2017-05-22 14:05:50,860 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:05:50,860 Stage-1 map = 13%,  reduce = 0%, Cumulative CPU 18.18 sec
2017-05-22 14:05:51,891 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:05:51,890 Stage-1 map = 44%,  reduce = 0%, Cumulative CPU 65.4 sec
2017-05-22 14:05:59,080 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:05:59,080 Stage-1 map = 50%,  reduce = 0%, Cumulative CPU 74.49 sec
2017-05-22 14:06:00,108 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:06:00,107 Stage-1 map = 63%,  reduce = 0%, Cumulative CPU 92.57 sec
2017-05-22 14:06:01,131 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:06:01,131 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 129.85 sec
2017-05-22 14:06:06,253 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:06:06,252 Stage-1 map = 94%,  reduce = 0%, Cumulative CPU 135.96 sec
2017-05-22 14:06:09,329 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : 2017-05-22 14:06:09,329 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 144.8 sec
2017-05-22 14:06:10,355 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : MapReduce Total cumulative CPU time: 2 minutes 24 seconds 800 msec
2017-05-22 14:06:10,380 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Ended Job = job_1493346400817_0062
2017-05-22 14:06:10,395 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Loading data to table default.kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d
2017-05-22 14:06:10,733 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Table default.kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d stats: [numFiles=16, numRows=25452592, totalSize=371258175, rawDataSize=1466297840]
2017-05-22 14:06:10,734 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : MapReduce Jobs Launched: 
2017-05-22 14:06:10,734 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Stage-Stage-1: Map: 16   Cumulative CPU: 144.8 sec   HDFS Read: 4172291379 HDFS Write: 371260334 SUCCESS
2017-05-22 14:06:10,735 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Total MapReduce CPU Time Spent: 2 minutes 24 seconds 800 msec
2017-05-22 14:06:10,735 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : OK
2017-05-22 14:06:10,741 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] hive.CreateFlatHiveTableStep:38 : Time taken: 37.045 seconds
2017-05-22 14:06:11,112 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-316] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-00
2017-05-22 14:06:11,116 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-316] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-00
2017-05-22 14:06:11,118 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-316] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-00
2017-05-22 14:06:11,120 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-00 from RUNNING to SUCCEED
2017-05-22 14:06:11,121 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-316] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:06:11,126 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-316] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:06:11,128 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-316] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from RUNNING to READY
2017-05-22 14:06:11,145 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:114 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} prepare to schedule
2017-05-22 14:06:11,145 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:117 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} scheduled
2017-05-22 14:06:11,146 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:110 : Executing AbstractExecutable (cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36)
2017-05-22 14:06:11,147 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:06:11,148 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 0 should running, 1 actual running, 0 stopped, 1 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:06:11,149 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from READY to RUNNING
2017-05-22 14:06:11,150 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:110 : Executing AbstractExecutable (Redistribute Flat Hive Table)
2017-05-22 14:06:11,150 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-01
2017-05-22 14:06:11,152 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-01 from READY to RUNNING
2017-05-22 14:06:11,355 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] hive.metastore:386 : Trying to connect to metastore with URI thrift://gp-node-master.zoldbs.com.cn:9083
2017-05-22 14:06:11,379 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] hive.metastore:431 : Opened a connection to metastore, current connections: 1
2017-05-22 14:06:11,404 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] hive.metastore:483 : Connected to metastore.
2017-05-22 14:06:11,456 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:301 : Row count of table 'kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d' is 25452592
2017-05-22 14:06:11,457 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : total input rows = 25452592
2017-05-22 14:06:11,457 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : expected input rows per mapper = 20000000
2017-05-22 14:06:11,457 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : num reducers for RedistributeFlatHiveTableStep = 1
2017-05-22 14:06:11,457 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Redistribute table, cmd: 
2017-05-22 14:06:11,457 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : hive -e "USE default;
SET dfs.replication=2;
SET hive.exec.compress.output=true;
SET hive.auto.convert.join=true;
SET hive.auto.convert.join.noconditionaltask=true;
SET hive.auto.convert.join.noconditionaltask.size=100000000;
SET mapreduce.job.split.metainfo.maxsize=-1;
SET hive.stats.autogather=true;
SET hive.merge.mapfiles=false;
SET hive.merge.mapredfiles=false;
SET mapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;
SET mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.SnappyCodec;

set mapreduce.job.reduces=1;

set hive.merge.mapredfiles=false;

INSERT OVERWRITE TABLE kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d SELECT * FROM kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d DISTRIBUTE BY RAND();

"
2017-05-22 14:06:11,587 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
2017-05-22 14:06:14,690 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
2017-05-22 14:06:15,145 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : SLF4J: Class path contains multiple SLF4J bindings.
2017-05-22 14:06:15,145 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2017-05-22 14:06:15,146 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : SLF4J: Found binding in [jar:file:/opt/cloudera/apache-kylin-2.0.0-bin/spark/lib/spark-assembly-1.6.3-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
2017-05-22 14:06:15,146 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2017-05-22 14:06:15,148 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2017-05-22 14:06:15,463 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 17/05/22 14:06:15 WARN mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present.  Continuing without it.
2017-05-22 14:06:15,967 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
2017-05-22 14:06:17,301 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 
2017-05-22 14:06:17,301 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/jars/hive-common-1.1.0-cdh5.9.0.jar!/hive-log4j.properties
2017-05-22 14:06:19,122 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : OK
2017-05-22 14:06:19,123 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Time taken: 0.633 seconds
2017-05-22 14:06:19,660 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Query ID = root_20170522140606_aa63fd8c-270c-4849-87bd-5a9bd9667d26
2017-05-22 14:06:19,660 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Total jobs = 1
2017-05-22 14:06:19,675 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Launching Job 1 out of 1
2017-05-22 14:06:19,680 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Number of reduce tasks not specified. Defaulting to jobconf value of: 1
2017-05-22 14:06:19,680 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : In order to change the average load for a reducer (in bytes):
2017-05-22 14:06:19,680 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 :   set hive.exec.reducers.bytes.per.reducer=<number>
2017-05-22 14:06:19,680 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : In order to limit the maximum number of reducers:
2017-05-22 14:06:19,681 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 :   set hive.exec.reducers.max=<number>
2017-05-22 14:06:19,681 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : In order to set a constant number of reducers:
2017-05-22 14:06:19,681 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 :   set mapreduce.job.reduces=<number>
2017-05-22 14:06:21,232 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Starting Job = job_1493346400817_0063, Tracking URL = http://gp-node-master.zoldbs.com.cn:8088/proxy/application_1493346400817_0063/
2017-05-22 14:06:21,232 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Kill Command = /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hadoop/bin/hadoop job  -kill job_1493346400817_0063
2017-05-22 14:06:25,528 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Hadoop job information for Stage-1: number of mappers: 6; number of reducers: 1
2017-05-22 14:06:25,596 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:25,591 Stage-1 map = 0%,  reduce = 0%
2017-05-22 14:06:25,705 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:06:38,988 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:38,987 Stage-1 map = 34%,  reduce = 0%, Cumulative CPU 78.23 sec
2017-05-22 14:06:40,014 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:40,014 Stage-1 map = 40%,  reduce = 0%, Cumulative CPU 79.6 sec
2017-05-22 14:06:42,067 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:42,067 Stage-1 map = 68%,  reduce = 0%, Cumulative CPU 97.12 sec
2017-05-22 14:06:43,093 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:43,092 Stage-1 map = 79%,  reduce = 0%, Cumulative CPU 99.61 sec
2017-05-22 14:06:47,196 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:47,195 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 113.55 sec
2017-05-22 14:06:48,224 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:48,223 Stage-1 map = 90%,  reduce = 0%, Cumulative CPU 116.72 sec
2017-05-22 14:06:54,377 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:54,377 Stage-1 map = 92%,  reduce = 0%, Cumulative CPU 124.59 sec
2017-05-22 14:06:57,456 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:57,456 Stage-1 map = 96%,  reduce = 0%, Cumulative CPU 130.1 sec
2017-05-22 14:06:59,511 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:06:59,511 Stage-1 map = 96%,  reduce = 28%, Cumulative CPU 142.4 sec
2017-05-22 14:07:00,542 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:07:00,541 Stage-1 map = 100%,  reduce = 28%, Cumulative CPU 146.07 sec
2017-05-22 14:07:05,686 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:07:05,686 Stage-1 map = 100%,  reduce = 40%, Cumulative CPU 150.69 sec
2017-05-22 14:07:08,768 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:07:08,767 Stage-1 map = 100%,  reduce = 51%, Cumulative CPU 153.86 sec
2017-05-22 14:07:11,856 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:07:11,856 Stage-1 map = 100%,  reduce = 61%, Cumulative CPU 157.0 sec
2017-05-22 14:07:14,943 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:07:14,942 Stage-1 map = 100%,  reduce = 67%, Cumulative CPU 168.74 sec
2017-05-22 14:07:25,707 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:08:06,061 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : 2017-05-22 14:08:06,060 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 224.04 sec
2017-05-22 14:08:07,084 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : MapReduce Total cumulative CPU time: 3 minutes 44 seconds 40 msec
2017-05-22 14:08:07,108 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Ended Job = job_1493346400817_0063
2017-05-22 14:08:07,123 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Loading data to table default.kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d
2017-05-22 14:08:07,390 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Table default.kylin_intermediate_cc_ab29c420_a0e7_417a_b067_c6c20039a30d stats: [numFiles=1, numRows=25452592, totalSize=371918513, rawDataSize=1466297840]
2017-05-22 14:08:07,391 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : MapReduce Jobs Launched: 
2017-05-22 14:08:07,392 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Stage-Stage-1: Map: 6  Reduce: 1   Cumulative CPU: 224.04 sec   HDFS Read: 371293020 HDFS Write: 371918651 SUCCESS
2017-05-22 14:08:07,392 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Total MapReduce CPU Time Spent: 3 minutes 44 seconds 40 msec
2017-05-22 14:08:07,392 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : OK
2017-05-22 14:08:07,396 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.AbstractExecutable:38 : Time taken: 108.265 seconds
2017-05-22 14:08:07,799 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-01
2017-05-22 14:08:07,802 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-01
2017-05-22 14:08:07,805 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-01
2017-05-22 14:08:07,807 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-01 from RUNNING to SUCCEED
2017-05-22 14:08:07,808 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:08:07,814 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-320] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:08:07,815 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-320] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from RUNNING to READY
2017-05-22 14:08:07,832 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:114 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} prepare to schedule
2017-05-22 14:08:07,833 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:117 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} scheduled
2017-05-22 14:08:07,833 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] execution.AbstractExecutable:110 : Executing AbstractExecutable (cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36)
2017-05-22 14:08:07,834 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:08:07,835 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 0 should running, 1 actual running, 0 stopped, 1 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:08:07,836 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from READY to RUNNING
2017-05-22 14:08:07,838 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] execution.AbstractExecutable:110 : Executing AbstractExecutable (Extract Fact Table Distinct Columns)
2017-05-22 14:08:07,839 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:08:07,840 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-02 from READY to RUNNING
2017-05-22 14:08:07,898 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.MapReduceExecutable:115 : parameters of the MapReduceExecutable:
2017-05-22 14:08:07,898 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.MapReduceExecutable:116 :  -conf /opt/cloudera/kylin/conf/kylin_job_conf.xml -cubename cc -output hdfs:///kylin/kylin_metadata/kylin-df682969-46e7-4f05-9394-0cbee8247e54/cc/fact_distinct_columns -segmentid ab29c420-a0e7-417a-b067-c6c20039a30d -statisticsenabled true -statisticsoutput hdfs:///kylin/kylin_metadata/kylin-df682969-46e7-4f05-9394-0cbee8247e54/cc/fact_distinct_columns/statistics -statisticssamplingpercent 100 -jobname Kylin_Fact_Distinct_Columns_cc_Step -cubingJobId df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:08:07,948 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] steps.FactDistinctColumnsJob:106 : Starting: Kylin_Fact_Distinct_Columns_cc_Step
2017-05-22 14:08:07,948 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:163 : append job jar: /opt/cloudera/kylin/lib/kylin-job-2.0.0.jar
2017-05-22 14:08:07,948 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:171 : append kylin.hbase.dependency: /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hbase/bin/../lib/hbase-common-1.2.0-cdh5.9.0.jar to mapreduce.application.classpath
2017-05-22 14:08:07,948 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:188 : Hadoop job classpath is: $HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,$MR2_CLASSPATH,/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hbase/bin/../lib/hbase-common-1.2.0-cdh5.9.0.jar
2017-05-22 14:08:07,949 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:200 : Hive Dependencies Before Filtered: /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive/conf,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-httpclient-3.0.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hbase-server.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-configuration-1.6.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/apache-log4j-extras-1.2.17.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/geronimo-jta_1.1_spec-1.1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/accumulo-fate-1.6.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-digester-1.8.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/junit-4.11.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-service-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-exec.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jetty-all-server-7.6.0.v20120127.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims-scheduler-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/maven-scm-api-1.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/parquet-hadoop-bundle.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/accumulo-start-1.6.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/datanucleus-rdbms-3.2.9.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-dbcp-1.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/bonecp-0.8.0.RELEASE.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jsp-api-2.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-hwi-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-collections-3.2.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-cli.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/htrace-core.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-beeline-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hbase-hadoop2-compat.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/findbugs-annotations-1.3.9-1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hbase-common.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jpam-1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/opencsv-2.3.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-testutils-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/snappy-java-1.0.4.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/libfb303-0.9.3.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-contrib.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/stringtemplate-3.2.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/gson-2.2.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-math-2.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jetty-all-7.6.0.v20120127.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/asm-commons-3.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/metrics-json-3.0.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/curator-client-2.6.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-exec-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims-0.23-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/metrics-jvm-3.0.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/avro.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-io-2.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims-common-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/joda-time-1.6.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/logredactor-1.0.3.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/asm-3.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-beeline.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/httpcore-4.2.5.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-service.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jackson-jaxrs-1.9.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/activation-1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-compress-1.4.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jcommander-1.32.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-codec-1.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hbase-annotations.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jta-1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-jdbc-standalone.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-logging-1.1.3.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jackson-xc-1.9.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jasper-compiler-5.5.23.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/paranamer-2.3.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/ant-1.9.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-ant-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hbase-hadoop-compat.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hbase-protocol.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-serde.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/curator-recipes-2.6.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/asm-tree-3.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims-scheduler.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-common.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/ant-launcher-1.9.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/mockito-all-1.9.5.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-hbase-handler-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jasper-runtime-5.5.23.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-beanutils-core-1.8.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jersey-servlet-1.14.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/curator-framework-2.6.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-metastore-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-jdbc-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/log4j-1.2.16.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-hbase-handler.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jamon-runtime-2.3.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-hwi.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jackson-databind-2.2.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-ant.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-accumulo-handler-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/antlr-2.7.7.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-metastore.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-jdbc-1.1.0-cdh5.9.0-standalone.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-lang3-3.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jackson-core-2.2.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/ST4-4.0.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-pool-1.5.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jdo-api-3.0.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/plexus-utils-1.5.6.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/accumulo-trace-1.6.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-vfs2-2.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/antlr-runtime-3.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/tempus-fugit-1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-serde-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-compiler-2.7.6.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/libthrift-0.9.3.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jsr305-3.0.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/maven-scm-provider-svnexe-1.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/httpclient-4.2.5.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-accumulo-handler.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims-0.23.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-beanutils-1.9.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-jdbc.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jackson-annotations-2.2.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/groovy-all-2.4.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-el-1.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/super-csv-2.2.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/regexp-1.3.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/datanucleus-api-jdo-3.2.6.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/metrics-core-3.0.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-contrib-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/xz-1.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/eigenbase-properties-1.1.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/guava-14.0.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-cli-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/mail-1.4.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hamcrest-core-1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-shims-common.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/maven-scm-provider-svn-commons-1.4.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/velocity-1.5.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/high-scale-lib-1.1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-common-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/derby-10.11.1.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/accumulo-core-1.6.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/servlet-api-2.5.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hbase-client.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/zookeeper.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/janino-2.7.6.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/stax-api-1.0.1.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jersey-server-1.14.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/oro-2.0.8.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-cli-1.2.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/datanucleus-core-3.2.10.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-testutils.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/geronimo-jaspic_1.0_spec-1.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/commons-lang-2.6.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/jline-2.12.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-1.1.0-cdh5.9.0.jar
2017-05-22 14:08:08,035 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:202 : Hive Dependencies After Filtered: /opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-exec.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-exec-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-metastore-1.1.0-cdh5.9.0.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive/lib/hive-metastore.jar,/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/bin/../lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-1.1.0-cdh5.9.0.jar
2017-05-22 14:08:08,035 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:230 : Kafka Dependencies: 
2017-05-22 14:08:08,092 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:358 : Job 'tmpjars' updated -- file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive/lib/hive-exec.jar,file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive/lib/hive-exec-1.1.0-cdh5.9.0.jar,file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive/lib/hive-metastore-1.1.0-cdh5.9.0.jar,file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive/lib/hive-metastore.jar,file:/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p0.23/lib/hive-hcatalog/share/hcatalog/hive-hcatalog-core-1.1.0-cdh5.9.0.jar
2017-05-22 14:08:08,273 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] hive.metastore:386 : Trying to connect to metastore with URI thrift://gp-node-master.zoldbs.com.cn:9083
2017-05-22 14:08:08,274 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] hive.metastore:431 : Opened a connection to metastore, current connections: 2
2017-05-22 14:08:08,275 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] hive.metastore:483 : Connected to metastore.
2017-05-22 14:08:08,467 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.KylinConfig:131 : The URI /opt/cloudera/kylin/bin/../tomcat/temp/kylin_job_meta145248982383110119/meta is recognized as LOCAL_FOLDER
2017-05-22 14:08:08,467 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.KylinConfigBase:153 : Kylin Config was updated with kylin.metadata.url : /opt/cloudera/kylin/bin/../tomcat/temp/kylin_job_meta145248982383110119/meta
2017-05-22 14:08:08,467 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] persistence.ResourceStore:88 : Using metadata url /opt/cloudera/kylin/bin/../tomcat/temp/kylin_job_meta145248982383110119/meta for resource store
2017-05-22 14:08:08,470 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:555 : Dump resources to /opt/cloudera/kylin/bin/../tomcat/temp/kylin_job_meta145248982383110119/meta took 4 ms
2017-05-22 14:08:08,470 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:513 : HDFS meta dir is: file:///opt/cloudera/kylin/bin/../tomcat/temp/kylin_job_meta145248982383110119/meta
2017-05-22 14:08:08,470 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:372 : Job 'tmpfiles' updated -- file:///opt/cloudera/kylin/bin/../tomcat/temp/kylin_job_meta145248982383110119/meta
2017-05-22 14:08:08,470 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] Configuration.deprecation:1174 : mapred.output.compress is deprecated. Instead, use mapreduce.output.fileoutputformat.compress
2017-05-22 14:08:08,525 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] client.RMProxy:98 : Connecting to ResourceManager at gp-node-master.zoldbs.com.cn/10.19.8.102:8032
2017-05-22 14:08:09,716 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] Configuration.deprecation:1174 : mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
2017-05-22 14:08:09,733 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] mapred.FileInputFormat:249 : Total input paths to process : 1
2017-05-22 14:08:09,771 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] mapreduce.JobSubmitter:202 : number of splits:3
2017-05-22 14:08:09,801 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] mapreduce.JobSubmitter:291 : Submitting tokens for job: job_1493346400817_0064
2017-05-22 14:08:09,946 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] impl.YarnClientImpl:260 : Submitted application application_1493346400817_0064
2017-05-22 14:08:09,983 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] mapreduce.Job:1311 : The url to track the job: http://gp-node-master.zoldbs.com.cn:8088/proxy/application_1493346400817_0064/
2017-05-22 14:08:09,983 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.AbstractHadoopJob:520 : tempMetaFileString is : file:///opt/cloudera/kylin/bin/../tomcat/temp/kylin_job_meta145248982383110119/meta
2017-05-22 14:08:09,990 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:08:20,082 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:08:20,085 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:08:25,707 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:08:30,091 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:08:40,098 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:08:50,105 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:09:00,112 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:09:10,120 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:09:20,127 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:09:25,706 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:09:30,134 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:09:40,141 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:09:50,148 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:10:00,156 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:10:10,163 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:10:13,279 INFO  [HiveClientCache-cleaner-0] hive.metastore:512 : Closed a connection to metastore, current connections: 1
2017-05-22 14:10:20,171 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:10:25,709 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:10:30,178 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:10:40,185 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:10:50,193 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:11:00,201 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:11:10,209 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:11:20,216 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:11:25,711 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:11:30,222 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:11:40,230 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:11:50,237 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:12:00,244 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:12:10,251 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:12:20,257 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:12:25,708 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:12:30,265 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:12:40,272 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:12:50,279 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:13:00,285 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:13:10,292 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:13:20,298 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:13:25,706 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:13:30,305 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:13:40,313 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:13:50,321 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:14:00,327 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:14:10,333 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:14:20,341 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:14:25,705 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:14:30,348 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:14:40,356 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:14:50,363 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:15:00,371 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:15:10,377 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:15:14,953 DEBUG [http-bio-7070-exec-4] controller.UserController:64 : authentication.getPrincipal() is org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
2017-05-22 14:15:20,385 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:15:25,714 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:15:30,392 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:15:40,400 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:15:50,407 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:16:00,413 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:16:10,420 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:16:20,427 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:16:25,704 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:16:30,434 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:16:40,442 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:16:50,448 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:16:57,749 DEBUG [http-bio-7070-exec-10] controller.UserController:64 : authentication.getPrincipal() is org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
2017-05-22 14:16:58,736 DEBUG [http-bio-7070-exec-10] controller.UserController:64 : authentication.getPrincipal() is org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
2017-05-22 14:17:00,456 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:17:10,462 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:17:20,470 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:17:25,705 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:17:30,477 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:17:40,484 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:17:50,492 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:17:57,060 INFO  [http-bio-7070-exec-9] util.HBaseRegionSizeCalculator:78 : Calculating region sizes for table "KYLIN_7T62C0LXW8".
2017-05-22 14:18:00,500 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:18:10,508 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:18:20,516 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:18:25,706 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:18:30,523 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:18:30,637 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] common.HadoopCmdOutput:96 : Counters: 50
	File System Counters
		FILE: Number of bytes read=230609350
		FILE: Number of bytes written=347864381
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=372882718
		HDFS: Number of bytes written=227978384
		HDFS: Number of read operations=34
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=14
	Job Counters 
		Launched map tasks=3
		Launched reduce tasks=8
		Rack-local map tasks=3
		Total time spent by all maps in occupied slots (ms)=5950996
		Total time spent by all reduces in occupied slots (ms)=488360
		Total time spent by all map tasks (ms)=1487749
		Total time spent by all reduce tasks (ms)=122090
		Total vcore-seconds taken by all map tasks=1487749
		Total vcore-seconds taken by all reduce tasks=122090
		Total megabyte-seconds taken by all map tasks=24375279616
		Total megabyte-seconds taken by all reduce tasks=2000322560
	Map-Reduce Framework
		Map input records=25452592
		Map output records=152715741
		Map output bytes=1952996499
		Map output materialized bytes=115066266
		Input split bytes=28299
		Combine input records=175910325
		Combine output records=45773343
		Reduce input groups=21885363
		Reduce shuffle bytes=115066266
		Reduce input records=22578759
		Reduce output records=0
		Spilled Records=68352102
		Shuffled Maps =24
		Failed Shuffles=0
		Merged Map outputs=24
		GC time elapsed (ms)=21348
		CPU time spent (ms)=2017070
		Physical memory (bytes) snapshot=14984056832
		Virtual memory (bytes) snapshot=178878844928
		Total committed heap usage (bytes)=15523643392
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=0
	org.apache.kylin.engine.mr.steps.FactDistinctColumnsMapper$RawDataCounter
		BYTES=1491750432
2017-05-22 14:18:30,639 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:18:30,641 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:18:30,644 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-02
2017-05-22 14:18:30,645 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-02 from RUNNING to SUCCEED
2017-05-22 14:18:30,647 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:18:30,652 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-328] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:18:30,653 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-328] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from RUNNING to READY
2017-05-22 14:18:30,669 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:114 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} prepare to schedule
2017-05-22 14:18:30,670 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:117 : CubingJob{id=df682969-46e7-4f05-9394-0cbee8247e54, name=cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36, state=READY} scheduled
2017-05-22 14:18:30,670 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] execution.AbstractExecutable:110 : Executing AbstractExecutable (cc - 19700101000000_2922789940817071255 - BUILD - GMT+08:00 2017-05-22 14:04:36)
2017-05-22 14:18:30,671 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:18:30,672 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 0 should running, 1 actual running, 0 stopped, 1 ready, 36 already succeed, 10 error, 8 discarded, 0 others
2017-05-22 14:18:30,672 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from READY to RUNNING
2017-05-22 14:18:30,675 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] execution.AbstractExecutable:110 : Executing AbstractExecutable (Build Dimension Dictionary)
2017-05-22 14:18:30,675 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-03
2017-05-22 14:18:30,677 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-03 from READY to RUNNING
2017-05-22 14:18:30,698 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] common.HadoopShellExecutable:58 : parameters of the HadoopShellExecutable:
2017-05-22 14:18:30,698 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] common.HadoopShellExecutable:59 :  -cubename cc -segmentid ab29c420-a0e7-417a-b067-c6c20039a30d -input hdfs:///kylin/kylin_metadata/kylin-df682969-46e7-4f05-9394-0cbee8247e54/cc/fact_distinct_columns
2017-05-22 14:18:30,702 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] cli.DictionaryGeneratorCLI:62 : Building dictionary for DEFAULT.LOG_DATA_170416.IP
2017-05-22 14:18:31,088 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] steps.CreateDictionaryJob:94 : DictionaryProvider read dict from file: hdfs://gp-node-master.zoldbs.com.cn:8020/kylin/kylin_metadata/kylin-df682969-46e7-4f05-9394-0cbee8247e54/cc/fact_distinct_columns/LOG_DATA_170416.IP/IP.rldict-r-00000
2017-05-22 14:18:31,092 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] cli.DictionaryGeneratorCLI:67 : Dict for 'IP' has already been built, save it
2017-05-22 14:18:31,143 ERROR [Job df682969-46e7-4f05-9394-0cbee8247e54-460] common.HadoopShellExecutable:65 : error execute HadoopShellExecutable{id=df682969-46e7-4f05-9394-0cbee8247e54-03, name=Build Dimension Dictionary, state=RUNNING}
java.io.FileNotFoundException: File does not exist: /kylin/kylin_metadata/resources/dict/DEFAULT.LOG_DATA_170416/IP/56abfe1f-1fdf-4bec-baec-43721e693c32.dict
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1963)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1904)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1884)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1856)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:568)
	at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:89)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
	at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1280)
	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1265)
	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1253)
	at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:305)
	at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:271)
	at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:263)
	at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1584)
	at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:309)
	at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:305)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:305)
	at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:780)
	at org.apache.kylin.storage.hbase.HBaseResourceStore.getInputStream(HBaseResourceStore.java:207)
	at org.apache.kylin.storage.hbase.HBaseResourceStore.access$000(HBaseResourceStore.java:58)
	at org.apache.kylin.storage.hbase.HBaseResourceStore$2.visit(HBaseResourceStore.java:173)
	at org.apache.kylin.storage.hbase.HBaseResourceStore.visitFolder(HBaseResourceStore.java:143)
	at org.apache.kylin.storage.hbase.HBaseResourceStore.getAllResourcesImpl(HBaseResourceStore.java:168)
	at org.apache.kylin.common.persistence.ResourceStore.getAllResources(ResourceStore.java:182)
	at org.apache.kylin.common.persistence.ResourceStore.getAllResources(ResourceStore.java:175)
	at org.apache.kylin.dict.DictionaryManager.checkDupByInfo(DictionaryManager.java:374)
	at org.apache.kylin.dict.DictionaryManager.saveDictionary(DictionaryManager.java:318)
	at org.apache.kylin.cube.CubeManager.saveDictionary(CubeManager.java:234)
	at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:68)
	at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:54)
	at org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:66)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124)
	at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:64)
	at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:124)
	at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:142)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): File does not exist: /kylin/kylin_metadata/resources/dict/DEFAULT.LOG_DATA_170416/IP/56abfe1f-1fdf-4bec-baec-43721e693c32.dict
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1963)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1904)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1884)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1856)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:568)
	at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:89)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)

	at org.apache.hadoop.ipc.Client.call(Client.java:1471)
	at org.apache.hadoop.ipc.Client.call(Client.java:1408)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:230)
	at com.sun.proxy.$Proxy30.getBlockLocations(Unknown Source)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:256)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
	at com.sun.proxy.$Proxy31.getBlockLocations(Unknown Source)
	at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1278)
	... 34 more
2017-05-22 14:18:31,145 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-03
2017-05-22 14:18:31,148 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54-03
2017-05-22 14:18:31,149 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54-03 from RUNNING to ERROR
2017-05-22 14:18:31,152 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:18:31,154 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:18:31,155 DEBUG [Job df682969-46e7-4f05-9394-0cbee8247e54-460] dao.ExecutableDao:217 : updating job output, id: df682969-46e7-4f05-9394-0cbee8247e54
2017-05-22 14:18:31,157 INFO  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] execution.ExecutableManager:389 : job id:df682969-46e7-4f05-9394-0cbee8247e54 from RUNNING to ERROR
2017-05-22 14:18:31,157 WARN  [Job df682969-46e7-4f05-9394-0cbee8247e54-460] execution.AbstractExecutable:258 : no need to send email, user list is empty
2017-05-22 14:18:31,174 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 0 should running, 0 actual running, 0 stopped, 0 ready, 36 already succeed, 11 error, 8 discarded, 0 others
2017-05-22 14:19:25,706 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 0 should running, 0 actual running, 0 stopped, 0 ready, 36 already succeed, 11 error, 8 discarded, 0 others
2017-05-22 14:20:25,706 INFO  [pool-9-thread-1] threadpool.DefaultScheduler:124 : Job Fetcher: 0 should running, 0 actual running, 0 stopped, 0 ready, 36 already succeed, 11 error, 8 discarded, 0 others