[OOZIE-3146] Enhance local sharelib

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.0.0
• Component/s: core
• Assignee: Unassigned
• Reporter: Attila Sasvári
• Created: 21/Dec/17 18:43
• Updated: 21/Dec/17 18:43

Description

OOZIE-2601 made it possible to use local paths for the Oozie Sharelib.

Right now, ShareLibService checks the existence of files either on HDFS or local filesystem (symlinks followed too). It is actually checking the files or resolving symlinks in the Oozie server, while the actual files are in the task nodes.

The implementation could be improved. For example: we could introduce "a separate datastructure for local files and adding a new getShareLibJarsLocal(String actiontype) method,which will return the local files path as is (if it is a file with extension) or return both the path and /* appended to the path (/* works only for jars. The original directory path is required if there are config files under the directory). That can be then added directly to yarn.application.classpath setting."

Issue Links

• is related to: OOZIE-2601 Ability to use local paths for the sharelib