[KYLIN-1149] When yarn return an incomplete job tracking URL, Kylin will fail to get job status

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v0.7.1, v0.7.2, v1.0, v1.1, v1.1.1
• Fix Version/s: v1.2, v1.4.0
• Component/s: Job Engine
• Assignee: Shao Feng Shi
• Reporter: Shao Feng Shi
• Created: 16/Nov/15 07:42
• Updated: 22/Dec/15 09:20
• Resolved: 24/Nov/15 06:13

Description

When yarn return an incomplete job tracking URL, Kylin will fail to get job status, then the cube job will be in "Error" state:

[pool-9-thread-8]:[2015-11-16 04:58:32,062][ERROR][org.apache.kylin.engine.mr.common.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:91)] - error check status
java.lang.IllegalArgumentException: host parameter is null
    at org.apache.commons.httpclient.HttpConnection.setHost(HttpConnection.java:249)
    at org.apache.commons.httpclient.SimpleHttpConnectionManager.getConnectionWithTimeout(SimpleHttpConnectionManager.java:189)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:153)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
    at org.apache.kylin.engine.mr.common.HadoopStatusGetter.getHttpResponse(HadoopStatusGetter.java:77)
    at org.apache.kylin.engine.mr.common.HadoopStatusGetter.get(HadoopStatusGetter.java:54)
    at org.apache.kylin.engine.mr.common.HadoopStatusChecker.checkStatus(HadoopStatusChecker.java:56)
    at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:136)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
    at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
    at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
    at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:124)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Kylin should better check whether the URL is valid before visit it.