Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4408

allow jobs to set a JAR that is in the distributed cached

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.3, 2.0.0-alpha
    • Fix Version/s: 1.2.0, 2.0.2-alpha, 0.23.4
    • Component/s: mrv1, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Setting a job JAR with JobConf.setJar(String) and Job.setJar(String) assumes that the JAR is local to the client submitting the job, thus it triggers copying the JAR to HDFS and injecting it to the distributed cached.

      AFAIK, this is the only way to use uber JARs (JARs with JARs inside) in MR jobs.

      For jobs launched by Oozie, all JARs are already in HDFS. In order for Oozie to suport uber JARs (OOZIE-654) there should be a way for specifying as JAR a JAR that is already in HDFS.

      1. MAPREDUCE-4408.patch
        11 kB
        Robert Kanter
      2. MAPREDUCE-4408.patch
        11 kB
        Robert Kanter
      3. MAPREDUCE-4408-branch-1.patch
        8 kB
        Robert Kanter

        Issue Links

          Activity

            People

            • Assignee:
              Robert Kanter
              Reporter:
              Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development