[HADOOP-3298] Add support for transitive native library dependencies in DistributedCache

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate

Environment

unix (different handling would be required for windows)

• Assignee: Unassigned
• Reporter: Subramaniam Krishnan
• Created: 22/Apr/08 12:46
• Updated: 08/Jul/09 16:52
• Resolved: 22/Apr/08 13:03

Description

[Hadoop 1660] add the tasks's working directory to it's JAVA_LIBRARY_PATH, this allows to distribute JNI based native libs with the DistributedCache by symbolically linking them to the tasks's working directory.

There are many instances when the JNI components have transitive dependencies to other native libraries. Even if we have symlinks for these in tasks's working directory, they'll not resolved as transitive dependencies of native libraries are looked up in the shell's LD_LIBRARY_PATH and not in the JAVA_LIBRARY_PATH. This can be tackled by adding CWD to JAVA_LIBRARY_PATH in the hadoop-env.sh.
A better way to do this would be to add the CWD only in environment of the Task Tracker as only it needs it.

Issue Links

• is a clone of: HADOOP-1660 add support for native library toDistributedCache