[KNOX-857] Incorrect formation of <delete path> causes oozie mapreduce action failure

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Not A Bug
• Affects Version/s: 0.6.0
• Component/s: KnoxCLI
• Environment: HDP-2.4.0.0
• Assignee: Sumit Gupta
• Reporter: Kuldeep Kulkarni
• Created: 19/Jan/17 12:18
• Updated: 07/Nov/18 20:29
• Resolved: 24/Jan/17 17:27

Description

When Oozie workflow is submitted via Knox containing mapreduce-action. it always gets failed with below error:

Heart beat
{"properties":[{"key":"oozie.launcher.job.id","value":"job_1484748086731_0022","isFinal":false,"resource":"programatically"},{"key":"oozie.job.id","value":"0000017-170118140025343-oozie-oozi-W","isFinal":false,"resource":"programatically"},{"key":"oozie.action.id","value":"0000017-170118140025343-oozie-oozi-W@mr-node","isFinal":false,"resource":"programatically"},{"key":"mapreduce.job.tags","value":"oozie-20b9b1ecd11f08843b078e701ce5c1ad","isFinal":false,"resource":"programatically"}]}Starting the execution of prepare actions
Prepare execution in the Launcher Mapper has failed
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.MapReduceMain], exception invoking main(), java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
org.apache.oozie.action.hadoop.LauncherException: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:193)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: org.apache.oozie.action.hadoop.LauncherException: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
	at org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:77)
	at org.apache.oozie.action.hadoop.LauncherMapper.executePrepare(LauncherMapper.java:494)
	at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:189)
	... 8 more
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
	at org.apache.hadoop.fs.Path.initialize(Path.java:205)
	at org.apache.hadoop.fs.Path.<init>(Path.java:171)
	at org.apache.hadoop.fs.Path.<init>(Path.java:93)
	at org.apache.hadoop.fs.Globber.glob(Globber.java:211)
	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1655)
	at org.apache.oozie.action.hadoop.FSLauncherURIHandler.delete(FSLauncherURIHandler.java:59)
	at org.apache.oozie.action.hadoop.PrepareActionsDriver.execute(PrepareActionsDriver.java:90)
	at org.apache.oozie.action.hadoop.PrepareActionsDriver.doOperations(PrepareActionsDriver.java:68)
	... 10 more
Caused by: java.net.URISyntaxException: Expected scheme-specific part at index 5: hdfs:
	at java.net.URI$Parser.fail(URI.java:2848)
	at java.net.URI$Parser.failExpecting(URI.java:2854)
	at java.net.URI$Parser.parse(URI.java:3057)
	at java.net.URI.<init>(URI.java:746)
	at org.apache.hadoop.fs.Path.initialize(Path.java:202)
	... 17 more

Oozie Launcher failed, finishing Hadoop job gracefully

This happens because, <delete path> in the action configuration gets generated incorrectly.

    <delete path="hdfs://oozieknox2.openstacklocal:8020/user/ambari-qa/examples/output-data/hdfs://oozieknox2.openstacklocal:8020/map-reduce" />

Expected value is:

    <delete path="hdfs://oozieknox2.openstacklocal:8020/user/ambari-qa/examples/output-data/map-reduce" />

Command used to submit oozie job is:

[ambari-qa@oozieknox1 ~]$ curl -i -k -u ambari-qa:hadoop -H Content-Type:application/xml -T job.xml.bug -X POST 'https://oozieknox3.openstacklocal:8443/gateway/mytopology/oozie/v1/jobs?action=start'
HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Set-Cookie: JSESSIONID=13d8gzukx9aq019rdkb06li3pj;Path=/gateway/mytopology;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Server: Apache-Coyote/1.1
Date: Thu, 19 Jan 2017 12:06:39 GMT
Content-Type: application/json;charset=UTF-8
Content-Length: 45

Issue Links

• relates to: KNOX-862 Improve documentation of the Oozie service in the user guide