[FALCON-2261] Job submission fails with IllegalArgumentException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: trunk, 0.11
• Component/s: extensions
• Assignee: sandeep samudrala
• Reporter: Pragya Mittal
• Created: 20/Jan/17 17:21
• Updated: 08/Jun/17 12:18
• Resolved: 24/Jan/17 11:05

Description

I tried registering and submitting jobs via webhdfs and got the following error :

pragya.mittal@falcon1001:~$ falcon extension -register -extensionName example-1 -path webhdfs://192.168.138.236:14000/tmp/extensions/extension-example
Extension :example-1 registered successfully.

pragya.mittal@falcon1001:~$ falcon extension -submitAndSchedule -extensionName example-1 -jobName job-instance1 -file abc
Error in building the extension:java.lang.IllegalArgumentException: Wrong FS: webhdfs://192.168.138.236:14000/tmp/extensions/extension-example/libs/build, expected: hdfs://192.168.138.236:8020
	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193)
	at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118)
	at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114)
	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
	at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1970)
	at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1939)
	at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1915)
	at org.apache.falcon.ExtensionHandler.copyExtensionPackage(ExtensionHandler.java:163)
	at org.apache.falcon.ExtensionHandler.loadAndPrepare(ExtensionHandler.java:102)
	at org.apache.falcon.client.FalconClient.getEntities(FalconClient.java:1164)
	at org.apache.falcon.client.FalconClient.validateExtensionAndGetEntities(FalconClient.java:1130)
	at org.apache.falcon.client.FalconClient.getEntitiesForm(FalconClient.java:1099)
	at org.apache.falcon.client.FalconClient.submitAndScheduleExtensionJob(FalconClient.java:1180)
	at org.apache.falcon.cli.FalconExtensionCLI.extensionCommand(FalconExtensionCLI.java:127)
	at org.apache.falcon.cli.FalconCLI.run(FalconCLI.java:128)
	at org.apache.falcon.cli.FalconCLI.main(FalconCLI.java:62)

ERROR: Error in building the extension:Wrong FS: webhdfs://192.168.138.236:14000/tmp/extensions/extension-example/libs/build, expected: hdfs://192.168.138.236:8020

Issue Links

• relates to: FALCON-2182 Add support for Falcon user extensions
• links to: GitHub Pull Request #346