[MAPREDUCE-4082] hadoop-mapreduce-client-app's mrapp-generated-classpath file should not be in the module JAR

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 0.23.3, 2.0.2-alpha
• Component/s: build
• Hadoop Flags: Reviewed
• Assignee: Alejandro Abdelnur
• Reporter: Alejandro Abdelnur
• Created: 29/Mar/12 15:01
• Updated: 07/Sep/12 21:03
• Resolved: 29/Mar/12 21:51

Description

Currently the mrapp-generated-classpath file containing the 'built' classpath, which only makes sense during building/testing in the machine where the build happens, is bundled in the hadoop-mapreduce-client-app JAR.

Because the file is bundled in the hadoop-mapreduce-client-app JAR, its contents are added to the classpath of all MR jobs.

All this entries are useless and just pollute the classpath.

This file should not be bundled in the hadoop-mapreduce-client-app JAR.

As an example, the contents of this file in my local built are:

/Users/tucu/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar:/Users/tucu/.m2/repository/asm/asm/3.2/asm-3.2.jar:/Users/tucu/.m2/repository/com/cenqua/clover/clover/3.0.2/clover-3.0.2.jar:/Users/tucu/.m2/repository/com/google/guava/guava/r09/guava-r09.jar:/Users/tucu/.m2/repository/com/google/inject/guice/3.0/guice-3.0.jar:/Users/tucu/.m2/repository/com/google/inject/extensions/guice-servlet/3.0/guice-servlet-3.0.jar:/Users/tucu/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar:/Users/tucu/.m2/repository/com/googlecode/json-simple/json-simple/1.1/json-simple-1.1.jar:/Users/tucu/.m2/repository/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-client/1.8/jersey-client-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-grizzly2/1.8/jersey-grizzly2-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-json/1.8/jersey-json-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/contribs/jersey-guice/1.8/jersey-guice-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-core/1.8/jersey-test-framework-core-1.8.jar:/Users/tucu/.m2/repository/com/sun/jersey/jersey-test-framework/jersey-test-framework-grizzly2/1.8/jersey-test-framework-grizzly2-1.8.jar:/Users/tucu/.m2/repository/com/sun/xml/bind/jaxb-impl/2.2.3-1/jaxb-impl-2.2.3-1.jar:/Users/tucu/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/Users/tucu/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/Users/tucu/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/Users/tucu/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/Users/tucu/.m2/repository/commons-codec/commons-codec/1.4/commons-codec-1.4.jar:/Users/tucu/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/Users/tucu/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/Users/tucu/.m2/repository/commons-daemon/commons-daemon/1.0.3/commons-daemon-1.0.3.jar:/Users/tucu/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/Users/tucu/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/Users/tucu/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/Users/tucu/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/Users/tucu/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar:/Users/tucu/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/Users/tucu/.m2/repository/commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar:/Users/tucu/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar:/Users/tucu/.m2/repository/hsqldb/hsqldb/1.8.0.7/hsqldb-1.8.0.7.jar:/Users/tucu/.m2/repository/javax/activation/activation/1.1/activation-1.1.jar:/Users/tucu/.m2/repository/javax/inject/javax.inject/1/javax.inject-1.jar:/Users/tucu/.m2/repository/javax/servlet/servlet-api/2.5/servlet-api-2.5.jar:/Users/tucu/.m2/repository/javax/servlet/jsp/jsp-api/2.1/jsp-api-2.1.jar:/Users/tucu/.m2/repository/javax/xml/bind/jaxb-api/2.1/jaxb-api-2.1.jar:/Users/tucu/.m2/repository/jdiff/jdiff/1.0.9/jdiff-1.0.9.jar:/Users/tucu/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/Users/tucu/.m2/repository/junit/junit/4.8.2/junit-4.8.2.jar:/Users/tucu/.m2/repository/log4j/log4j/1.2.15/log4j-1.2.15.jar:/Users/tucu/.m2/repository/net/java/dev/jets3t/jets3t/0.6.1/jets3t-0.6.1.jar:/Users/tucu/.m2/repository/net/sf/kosmosfs/kfs/0.3/kfs-0.3.jar:/Users/tucu/.m2/repository/org/apache/avro/avro/1.5.4/avro-1.5.4.jar:/Users/tucu/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-annotations/0.23.1-cdh4b2-SNAPSHOT/hadoop-annotations-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-auth/0.23.1-cdh4b2-SNAPSHOT/hadoop-auth-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-common/0.23.1-cdh4b2-SNAPSHOT/hadoop-common-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-common/0.23.1-cdh4b2-SNAPSHOT/hadoop-common-0.23.1-cdh4b2-SNAPSHOT-tests.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-hdfs/0.23.1-cdh4b2-SNAPSHOT/hadoop-hdfs-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/0.23.1-cdh4b2-SNAPSHOT/hadoop-mapreduce-client-common-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/0.23.1-cdh4b2-SNAPSHOT/hadoop-mapreduce-client-core-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-shuffle/0.23.1-cdh4b2-SNAPSHOT/hadoop-mapreduce-client-shuffle-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-api/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-api-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-common/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-common-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-common/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-common-0.23.1-cdh4b2-SNAPSHOT-tests.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-server-common/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-server-common-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-server-nodemanager/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-server-nodemanager-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-server-resourcemanager/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-server-resourcemanager-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-server-resourcemanager/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-server-resourcemanager-0.23.1-cdh4b2-SNAPSHOT-tests.jar:/Users/tucu/.m2/repository/org/apache/hadoop/hadoop-yarn-server-web-proxy/0.23.1-cdh4b2-SNAPSHOT/hadoop-yarn-server-web-proxy-0.23.1-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/apache/zookeeper/zookeeper/3.4.3-cdh4b2-SNAPSHOT/zookeeper-3.4.3-cdh4b2-SNAPSHOT.jar:/Users/tucu/.m2/repository/org/aspectj/aspectjrt/1.6.5/aspectjrt-1.6.5.jar:/Users/tucu/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/Users/tucu/.m2/repository/org/codehaus/jackson/jackson-jaxrs/1.8.8/jackson-jaxrs-1.8.8.jar:/Users/tucu/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/Users/tucu/.m2/repository/org/codehaus/jackson/jackson-xc/1.8.8/jackson-xc-1.8.8.jar:/Users/tucu/.m2/repository/org/codehaus/jettison/jettison/1.1/jettison-1.1.jar:/Users/tucu/.m2/repository/org/eclipse/jdt/core/3.1.1/core-3.1.1.jar:/Users/tucu/.m2/repository/org/glassfish/javax.servlet/3.0/javax.servlet-3.0.jar:/Users/tucu/.m2/repository/org/glassfish/external/management-api/3.0.0-b012/management-api-3.0.0-b012.jar:/Users/tucu/.m2/repository/org/glassfish/gmbal/gmbal-api-only/3.0.0-b023/gmbal-api-only-3.0.0-b023.jar:/Users/tucu/.m2/repository/org/glassfish/grizzly/grizzly-framework/2.1.1/grizzly-framework-2.1.1-tests.jar:/Users/tucu/.m2/repository/org/glassfish/grizzly/grizzly-framework/2.1.1/grizzly-framework-2.1.1.jar:/Users/tucu/.m2/repository/org/glassfish/grizzly/grizzly-http/2.1.1/grizzly-http-2.1.1.jar:/Users/tucu/.m2/repository/org/glassfish/grizzly/grizzly-http-server/2.1.1/grizzly-http-server-2.1.1.jar:/Users/tucu/.m2/repository/org/glassfish/grizzly/grizzly-http-servlet/2.1.1/grizzly-http-servlet-2.1.1.jar:/Users/tucu/.m2/repository/org/glassfish/grizzly/grizzly-rcm/2.1.1/grizzly-rcm-2.1.1.jar:/Users/tucu/.m2/repository/org/jboss/netty/netty/3.2.3.Final/netty-3.2.3.Final.jar:/Users/tucu/.m2/repository/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5.jar:/Users/tucu/.m2/repository/org/mortbay/jetty/jetty/6.1.26/jetty-6.1.26.jar:/Users/tucu/.m2/repository/org/mortbay/jetty/jetty-util/6.1.26/jetty-util-6.1.26.jar:/Users/tucu/.m2/repository/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar:/Users/tucu/.m2/repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar:/Users/tucu/.m2/repository/org/xerial/snappy/snappy-java/1.0.3.2/snappy-java-1.0.3.2.jar:/Users/tucu/.m2/repository/oro/oro/2.0.8/oro-2.0.8.jar:/Users/tucu/.m2/repository/stax/stax-api/1.0.1/stax-api-1.0.1.jar:/Users/tucu/.m2/repository/tomcat/jasper-compiler/5.5.23/jasper-compiler-5.5.23.jar:/Users/tucu/.m2/repository/tomcat/jasper-runtime/5.5.23/jasper-runtime-5.5.23.jar:/Users/tucu/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar