[KYLIN-404] Can't get cube source record size.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v1.0, v1.4.0
• Labels: github-import
• Assignee: Shao Feng Shi
• Reporter: Luke Han
• Created: 09/Jan/15 16:05
• Updated: 06/Sep/15 07:11
• Resolved: 14/Aug/15 09:22

Description

Dear All!
I deploy kylin on Apache Hadoop, when i build the job , it failed on the last step. here is the some log message:
```java
[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,280][INFO][com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:93)] - cube_job_group.test_kylin_cube_without_slr_empty.30bbd69c-33ae-4983-aaf7-f46ab42b88dd.14 status: FINISHED
[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,280][DEBUG][com.kylinolap.cube.CubeManager.loadCubeInstance(CubeManager.java:604)] - Loading CubeInstance kylin_metadata_qa(key='/cube/test_kylin_cube_without_slr_empty.json')@kylin_metadata_qa@hbase:192.168.44.21:2181:/hbase9827
[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,295][DEBUG][com.kylinolap.common.persistence.ResourceStore.putResource(ResourceStore.java:166)] - Saving resource /job/30bbd69c-33ae-4983-aaf7-f46ab42b88dd (Store kylin_metadata_qa@hbase:192.168.44.21:2181:/hbase9827)
[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,300][INFO][com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:238)] - Updating cube segment FULL_BUILD for cube test_kylin_cube_without_slr_empty
[QuartzScheduler_Worker-4]:[2014-11-24 19:24:47,301][ERROR][com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:117)] - Can't get cube source record size.
java.lang.RuntimeException: Can't get cube source record size.
at com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:274)
at com.kylinolap.job.flow.JobFlowListener.jobWasExecuted(JobFlowListener.java:99)
at org.quartz.core.QuartzScheduler.notifyJobListenersWasExecuted(QuartzScheduler.java:1985)
at org.quartz.core.JobRunShell.notifyJobListenersComplete(JobRunShell.java:340)
at org.quartz.core.JobRunShell.run(JobRunShell.java:224)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
```
here is some version info:
Kylin 0.6.2
hadoop 2.4.1
hbase 0.98.4-hadoop2
zookeeper 3.4.5
hive 0.13.0

I find some config info on hbase
```
get 'kylin_metadata_qa_job', '/job/1eaf05f8-e9f5-4dd7-9748-ab9b3098a512'
{
"uuid": "1eaf05f8-e9f5-4dd7-9748-ab9b3098a512",
"name": "test_kylin_cube_without_slr_empty - FULL_BUILD - BUILD - PST 2014-11-24 23:00:15",
"type": "BUILD",
"duration": 461,
"steps": [

{ "name": "Create Intermediate Flat Hive Table", "info": null, "interruptCmd": null, "sequence_id": 0, "exec_cmd": "hive -e "DROPTABLEIFEXISTSkylin_intermediate_test_kylin_cube_without_slr_desc_FULL_BUILD_1eaf05f8_e9f5_4dd7_9748_ab9b3098a512;nCREATEEXTERNALTABLEIFNOTEXISTSkylin_intermediate_test_kylin_cube_without_slr_desc_FULL_BUILD_1eaf05f8_e9f5_4dd7_9748_ab9b3098a512n(nCAL_DTdaten, LEAF_CATEG_IDintn, LSTG_SITE_IDintn, META_CATEG_NAMEstringn, CATEG_LVL2_NAMEstringn, CATEG_LVL3_NAMEstringn, LSTG_FORMAT_NAMEstringn, SLR_SEGMENT_CDsmallintn, PRICEdecimal(38, 16)n, SELLER_IDbigintn) ...}

]
```
in source code: com.kylinolap.job.flow.JobFlowListener.updateCubeSegmentInfoOnSucceed(JobFlowListener.java:272)
```
JobStep createFlatTableStep = jobInstance.findStep(JobConstants.STEP_NAME_CREATE_FLAT_HIVE_TABLE);
if (null != createFlatTableStep) {
String sourceRecordsSize = createFlatTableStep.getInfo(JobInstance.SOURCE_RECORDS_SIZE);
if (sourceRecordsSize == null || sourceRecordsSize.equals(""))

{ throw new RuntimeException("Can't get cube source record size."); }

sourceSize = Long.parseLong(sourceRecordsSize);
} else

{ log.info("No step with name '" + JobConstants.STEP_NAME_CREATE_FLAT_HIVE_TABLE + "' is found"); }

```

the "info" is null, cased throw a RuntimeException, some one have idea with it??

---------------- Imported from GitHub ----------------
Url: https://github.com/KylinOLAP/Kylin/issues/101
Created by: Yancey1989
Labels: help wanted,
Assignee: binmahone
Created at: Tue Nov 25 14:22:13 CST 2014
State: closed