[MAPREDUCE-5960] JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0, 2.4.1
• Fix Version/s: 2.6.0
• Component/s: client
• Target Version/s: 2.6.0
• Hadoop Flags: Reviewed
• Assignee: Gera Shegalov
• Reporter: Gera Shegalov
• Created: 06/Jul/14 23:22
• Updated: 15/Dec/14 23:06
• Resolved: 06/Nov/14 15:18

Description

$ mapred job -submit myjob.xml 
2014-07-06 15:06:34.731 java[12120:1903] Unable to load realm info from SCDynamicStore
14/07/06 15:06:34 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/07/06 15:06:34 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/07/06 15:06:37 WARN conf.Configuration: bad conf file: element not <property>
14/07/06 15:06:37 WARN conf.Configuration: bad conf file: element not <property>
14/07/06 15:06:37 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/07/06 15:11:11 INFO mapreduce.JobSubmitter: Cleaning up the staging area /tmp/hadoop-yarn/staging/gshegalov/.staging/job_1404679996131_0004
Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: viewfs:/user/gshegalov/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar, expected: file:///
	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
	at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:80)
	at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:525)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:739)
	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:520)
	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1902)
	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1870)
	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1835)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyJar(JobSubmitter.java:286)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:254)
	at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
	at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
	at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
	at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
	at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:260)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1239)

Issue Links

• breaks: HADOOP-11409 FileContext.getFileContext can stack overflow if default fs misconfigured