Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4408

allow jobs to set a JAR that is in the distributed cached

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.3, 2.0.0-alpha
    • Fix Version/s: 1.2.0, 2.0.2-alpha, 0.23.4
    • Component/s: mrv1, mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Setting a job JAR with JobConf.setJar(String) and Job.setJar(String) assumes that the JAR is local to the client submitting the job, thus it triggers copying the JAR to HDFS and injecting it to the distributed cached.

      AFAIK, this is the only way to use uber JARs (JARs with JARs inside) in MR jobs.

      For jobs launched by Oozie, all JARs are already in HDFS. In order for Oozie to suport uber JARs (OOZIE-654) there should be a way for specifying as JAR a JAR that is already in HDFS.

        Attachments

        1. MAPREDUCE-4408.patch
          11 kB
          Robert Kanter
        2. MAPREDUCE-4408.patch
          11 kB
          Robert Kanter
        3. MAPREDUCE-4408-branch-1.patch
          8 kB
          Robert Kanter

          Issue Links

            Activity

              People

              • Assignee:
                rkanter Robert Kanter
                Reporter:
                tucu00 Alejandro Abdelnur
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: