[HADOOP-13410] RunJar adds the content of the jar twice to the classpath

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha1
• Component/s: util
• Assignee: Yuanbo Liu
• Reporter: Sangjin Lee
• Created: 22/Jul/16 23:44
• Updated: 31/Oct/16 22:37
• Resolved: 31/Oct/16 19:37

Description

Today when you run a "hadoop jar" command, the jar is unzipped to a temporary location and gets added to the classloader.

However, the original jar itself is still added to the classpath.

      List<URL> classPath = new ArrayList<>();
      classPath.add(new File(workDir + "/").toURI().toURL());
      classPath.add(file.toURI().toURL());
      classPath.add(new File(workDir, "classes/").toURI().toURL());
      File[] libs = new File(workDir, "lib").listFiles();
      if (libs != null) {
        for (File lib : libs) {
          classPath.add(lib.toURI().toURL());
        }
      }

As a result, the contents of the jar are present in the classpath twice and are completely redundant. Although this does not necessarily cause correctness issues, some stricter code written to require a single presence of files may fail.

I cannot think of a good reason why the jar should be added to the classpath if the unjarred content was added to it. I think we should remove the jar from the classpath.

Issue Links

• breaks: HADOOP-13620 Mapreduce job failure on submission
• relates to: HADOOP-13776 remove redundant classpath entries in RunJar
• relates to: HADOOP-12728 "hadoop jar my.jar" should probably prioritize my.jar in the classpath by default