Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
v3.0.1
-
None
-
None
-
EMR
Description
i am using kylin version:
Version: Apache kylin 3.0.1
Commit: 638a1eb68f257366d240105b33e5eea3bfa4dbf3
i have 30+ kylin cube build jobs but in a week one or two jobs are not getting registered in the cluster even though SCHEDULER tries to send jobs to the cluster,
its gets this error usually on step: Build N-Dimension Cuboid :
get following error:
// code placeholder 1113546:2022-05-20 00:26:04,873 ERROR [Scheduler 1065041011 Job 7df4f62c-b58f-b959-8ae9-ee0e5b2438fb-112] common.HadoopJobStatusChecker:58 : error check status 1113559:2022-05-20 00:26:04,907 ERROR [Scheduler 1065041011 Job 7df4f62c-b58f-b959-8ae9-ee0e5b2438fb-112] common.MapReduceExecutable:259 : error execute MapReduceExecutable{id=7df4f62c-b58f-b959-8ae9-ee0e5b2438fb-11, name=Build N-Dimension Cuboid : level 6, state=RUNNING} 2022-05-17 23:45:28,237 ERROR [Scheduler 1065041011 Job 4cc6e7c9-98e6-3f3c-db26-5e45e0455cab-129] common.MapReduceExecutable:259 : error execute MapReduceExecutable{id=4cc6e7c9-98e6-3f3c-db26-5e45e0455cab-05, name=Build Base Cuboid, state=RUNNING} java.lang.RuntimeException: org.apache.kylin.job.exception.PersistentException: java.io.IOException: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.kylin.job.execution.ExecutableManager.getOutput(ExecutableManager.java:178) at org.apache.kylin.job.execution.AbstractExecutable.getOutput(AbstractExecutable.java:389) at org.apache.kylin.job.execution.AbstractExecutable.isDiscarded(AbstractExecutable.java:515) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:179) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:179) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:114) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.kylin.job.exception.PersistentException: java.io.IOException: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. at org.apache.kylin.job.dao.ExecutableDao.getJobOutput(ExecutableDao.java:407) at org.apache.kylin.job.execution.ExecutableManager.getOutput(ExecutableManager.java:173) ... 10 more Caused by: java.io.IOException: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
Note: if i pause and resume the jobs it gets registerd and works fine.