Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
DEFAULT_YARN_APPLICATION_CLASSPATH are used by YARN projects to setup their AM / task classpaths if they have a dependency on Hadoop libraries.
It'll be useful to provide similar access to a Hadoop tarball (Hadoop libs, native libraries) etc, which could be used instead - for applications which do not want to rely upon Hadoop versions from a cluster node. This would also require functionality to update the classpath/env for the apps based on the structure of the tar.
As an example, MR has support for a full tar (for rolling upgrades). Similarly, Tez ships hadoop libraries along with it's build. I'm not sure about the Spark / Storm / HBase model for this - but using a common copy instead of everyone localizing Hadoop libraries would be useful.
Attachments
Issue Links
- relates to
-
MAPREDUCE-4421 Run MapReduce framework via the distributed cache
-
- Closed
-