[YARN-1601] 3rd party JARs are missing from hadoop-dist output

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0, 3.0.0-alpha1
• Fix Version/s: 2.3.0
• Hadoop Flags: Reviewed
• Assignee: Alejandro Abdelnur
• Reporter: Alejandro Abdelnur
• Created: 14/Jan/14 23:31
• Updated: 12/May/16 18:30
• Resolved: 15/Jan/14 17:01

Description

With the build changes of YARN-888 we are leaving out all 3rd party JArs used directly by YARN under /share/hadoop/yarn/lib/.

We did not notice this when running minicluster because they all happen to be in the classpath from hadoop-common and hadoop-yarn.

As 3d party JARs are not 'public' interfaces we cannot rely on them being provided to yarn by common and hdfs. (ie if common and hdfs stop using a 3rd party dependency that yarn uses this would break yarn if yarn does not pull that dependency explicitly).

Also, this will break bigtop hadoop build when they move to use branch-2 as they expect to find jars in /share/hadoop/yarn/lib/

Issue Links

• is broken by: YARN-888 clean up POM dependencies