[KYLIN-1319] Find a better way to check hadoop job status

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: v1.5.3
• Component/s: Job Engine
• Labels: newbie
• Assignee: Zhong Yanghong
• Reporter: liyang
• Created: 15/Jan/16 06:35
• Updated: 28/Aug/16 12:25
• Resolved: 17/May/16 01:51

Description

Currently Kylin retrieves jobs status via a resource manager web service like

https://<your_rm_server>:<port>/ws/v1/cluster/apps/${job_id}?anonymous=true

It is not most robust. Some user does not have "yarn.resourcemanager.webapp.address" set in yarm-site.xml, then get status will fail out-of-box. They have to set a Kylin property "kylin.job.yarn.app.rest.check.status.url" to overcome, which is not user friendly.

Kerberos authentication might cause problem too if security is enabled.

Is there a more robust way to check job status? Via Job API?

Issue Links

• relates to: KYLIN-1014 Support kerberos authentication while getting status from RM