[BIGTOP-3317] Oozie sharedlib create command fails when uploading to HDFS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0, 1.5.0
• Fix Version/s: 1.5.0, 1.4.1
• Assignee: Luca Toscano
• Reporter: Luca Toscano
• Created: 28/Feb/20 17:48
• Updated: 14/Mar/20 17:17
• Resolved: 10/Mar/20 03:00

Description

Hi!

I am testing oozie on BigTop 1.4, and when I execute the sharedlib create command I get the following error:

/usr/bin/oozie-setup sharelib create -fs hdfs://analytics-test-hadoop -locallib /usr/lib/oozie/oozie-sharelib.tar.gz

[..]

the destination path for sharelib is: /user/oozie/share/lib/lib_20200228153329
Error: No FileSystem for scheme: hdfs
Stack trace for the error was (for debug purposes):--------------------------------------java.io.IOException: No FileSystem for scheme: hdfs at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389) at org.apache.oozie.tools.OozieSharelibCLI.run(OozieSharelibCLI.java:172) at org.apache.oozie.tools.OozieSharelibCLI.main(OozieSharelibCLI.java:67)

After some debugging I found the following broken symlink:

file /usr/lib/oozie/lib/hadoop-hdfs.jar
/usr/lib/oozie/lib/hadoop-hdfs.jar: broken symbolic link to ../../hadoop/client/hadoop-hdfs.jar

The above is created by the oozie package, but the hadoop-hdfs.jar seems not present. I removed it and added the following in /usr/lib/oozie/lib:

ln -s ../../hadoop/client/hadoop-hdfs-client.jar hadoop-hdfs-client.jar

And now sharedlib upgrade seems working. It should be an easy fix in the deb packaging of oozie if what I wrote is correct (I can send a pull request in case).

Issue Links

• links to: GitHub Pull Request #610