Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
v2.1.0
-
None
-
CDH5.7
Description
在CDH5.7环境下新搭建kylin2.1环境
1.需要修改队列名称，具体操作如下
在kylin.properties文件配置队列参数如下：
kylin.source.kylin.client=beeline
kylin.engine.mr.config-override.mapreduce.job.queuename=cbasQueue
kylin.source.hive.config-override.mapreduce.job.queuename=cbasQueue
在kylin_hive_conf.xml里面配
mapred.job.queue.name
在
kylin_job_conf_inmem.xml
kylin_job_conf.xml文件里配
mapreduce.job.queuename=cbasQueue
2.修改好配置后重启kylin，在load hive table时，运行的mr任务报错，详情如下：
User: bdcbas1234
Name: Kylin Hive Column Cardinality Job table=CBAS_DB.S_WEB_APP_PAGE output=hdfs://zhcbdpII/bdcbasApp/kylin/bdcbasApp-kylin_metadata/cardinality/5ae93815-0f53-412e-bb53-07c194c873d7/CBAS_DB.S_WEB_APP_PAGE
Application Type: MAPREDUCE
Application Tags:
State: FAILED
FinalStatus: FAILED
Started: 星期二 十月 17 16:25:47 +0800 2017
Elapsed: 8sec
Tracking URL: History
Diagnostics:
Application application_1507516583223_4851 failed 2 times due to AM Container for appattempt_1507516583223_4851_000002 exited with exitCode: 1 due to: Exception from container-launch.
Container id: container_1507516583223_4851_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Shell output: main : command provided 1
main : user is bdcbas1234
main : requested yarn user is bdcbas1234
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
3.在BULID CUBE时，在第2步失败：
#1 Step Name: Create Intermediate Flat Hive Table
执行成功，mr任务使用的queue为配置后的cbasQueue
#2 Step Name: Redistribute Flat Hive Table
执行失败，mr任务select count
from ...6d_9935_49f0c9600e38(Stage-1)使用的queue为配置前的default
4.将kylin.properties文件里的参数
kylin.source.kylin.client修改为cli
执行load hive table错误与beeline相同
5.在BULID CUBE时,在第3步失败
#1 Step Name: Create Intermediate Flat Hive Table
执行成功，与beeline相同
#2 Step Name: Redistribute Flat Hive Table
执行成功，后台没有运行mr任务，kylin日志打印如下
Row count is 0, no need to redistribute
#3 Step Name: Extract Fact Table Distinct Columns
执行失败，日志如下：
User: bdcbas1234
Name: Kylin_Fact_Distinct_Columns_w_cub_source_Step
Application Type: MAPREDUCE
Application Tags:
State: FAILED
FinalStatus: FAILED
Started: 星期二 十月 17 13:52:40 +0800 2017
Elapsed: 26sec
Tracking URL: History
Diagnostics:
Application application_1507516583223_4639 failed 2 times due to AM Container for appattempt_1507516583223_4639_000002 exited with exitCode: 1 due to: Exception from container-launch.
Container id: container_1507516583223_4639_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:290)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Shell output: main : command provided 1
main : user is bdcbas1234
main : requested yarn user is bdcbas1234
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.