[TEZ-2931] Running DistCp job in Tez with DEBUG logging fails

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.5.2
• Target Version/s: 0.8.6
• Assignee: Unassigned
• Reporter: Prasanth Jayachandran
• Created: 07/Nov/15 02:06
• Updated: 14/Mar/17 03:40

Description

When working on HIVE-12364 I noticed the following exception when tez DEBUG logging is enabled. This fails all hive jobs that use distcp

Caused by: java.io.IOException: Cannot execute DistCp process: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: application_1446860311492_0003-distcp:%20insert%20overwrite%20directory%20'/tmp/src'%20...src(Stage-1)-tez-dag.pb.txt
        at org.apache.hadoop.hive.shims.Hadoop23Shims.runDistCp(Hadoop23Shims.java:1180)
        at org.apache.hadoop.hive.common.FileUtils.copy(FileUtils.java:554)
        at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2625)
        ... 36 more
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: application_1446860311492_0003-distcp:%20insert%20overwrite%20directory%20'/tmp/src'%20...src(Stage-1)-tez-dag.pb.txt
        at org.apache.hadoop.fs.Path.initialize(Path.java:206)
        at org.apache.hadoop.fs.Path.<init>(Path.java:172)
        at org.apache.hadoop.fs.Path.<init>(Path.java:94)
        at org.apache.tez.common.TezCommonUtils.getTezTextPlanStagingPath(TezCommonUtils.java:207)
        at org.apache.tez.client.TezClientUtils.localizeDagPlanAsText(TezClientUtils.java:737)
        at org.apache.tez.client.TezClientUtils.createApplicationSubmissionContext(TezClientUtils.java:598)
        at org.apache.tez.client.TezClient.submitDAGApplication(TezClient.java:680)
        at org.apache.tez.client.MRTezClient.submitDAGApplication(MRTezClient.java:47)
        at org.apache.tez.mapreduce.client.YARNRunner.submitJob(YARNRunner.java:648)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:536)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
        at org.apache.hadoop.tools.DistCp.execute(DistCp.java:162)
        at org.apache.hadoop.hive.shims.Hadoop23Shims.runDistCp(Hadoop23Shims.java:1177)
        ... 38 more
Caused by: java.net.URISyntaxException: Relative path in absolute URI: application_1446860311492_0003-distcp:%20insert%20overwrite%20directory%20'/tmp/src'%20...src(Stage-1)-tez-dag.pb.txt
        at java.net.URI.checkPath(URI.java:1823)
        at java.net.URI.<init>(URI.java:745)
        at org.apache.hadoop.fs.Path.initialize(Path.java:203)
        ... 55 more

Issue Links

• breaks: HIVE-12364 Distcp job fails when run under Tez