[MAPREDUCE-4408] allow jobs to set a JAR that is in the distributed cached

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.3, 2.0.0-alpha
• Fix Version/s: 1.2.0, 2.0.2-alpha, 0.23.4
• Component/s: mrv1, mrv2
• Hadoop Flags: Reviewed
• Assignee: Robert Kanter
• Reporter: Alejandro Abdelnur
• Created: 06/Jul/12 23:53
• Updated: 11/Oct/12 17:48
• Resolved: 24/Aug/12 23:44

Description

Setting a job JAR with JobConf.setJar(String) and Job.setJar(String) assumes that the JAR is local to the client submitting the job, thus it triggers copying the JAR to HDFS and injecting it to the distributed cached.

AFAIK, this is the only way to use uber JARs (JARs with JARs inside) in MR jobs.

For jobs launched by Oozie, all JARs are already in HDFS. In order for Oozie to suport uber JARs (OOZIE-654) there should be a way for specifying as JAR a JAR that is already in HDFS.

Issue Links

• is required by: OOZIE-654 Provide a way to use 'uber' jars with Oozie MR actions
• requires: MAPREDUCE-4068 Jars in lib subdirectory of the submittable JAR are not added to the classpath