[OOZIE-2512] ShareLibservice returns incorrect path for jar

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.3.0
• Assignee: Satish Saley
• Reporter: Satish Saley
• Created: 21/Apr/16 17:55
• Updated: 02/Dec/16 21:02
• Resolved: 12/Aug/16 04:28

Description

If we have oozie.service.ShareLibService.mapping.file setting pointing to a metafile, then ShareLibServe loads paths from that file.

We can mention path to a directory or path to a direct file.

When path to a directory is mentioned, the paths are populated correctly, but not when we mentioned direct path for a file.

Consider following paths:

• /sharelib/pig/
  • pig.jar
  • some.jar
• /sharelib/spark
  • spark-assembly.jar

In metafile, we have

oozie.pig=/sharelib/pig/
oozie.spark=/sharelib/spark/spark-assembly.jar

Now, the SharelibService calculates the paths as
pig - hdfs://clustername.com:8020/sharelib/pig/pig.jar,hdfs://clustername.com:8020/sharelib/pig/some.jar
spark - /sharelib/spark/spark-assembly.jar

The spark path does not have hdfs prefixed.

Later on, when we run a spark action, it fails with

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, File file:/sharelib/spark/spark-assembly.jar does not exist
java.io.FileNotFoundException: File file:/sharelib/spark/spark-assembly.jar does not exist

Remember, if we already mentioned hdfs prefixed paths in metafile, then it works fine.