[SPARK-3122] hadoop-yarn dependencies cannot be resolved

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.0.0, 1.0.2
• Component/s: YARN
• Labels: build easyfix hadoop maven

Environment

spark 1.0.1 + YARN + hadoop 2.4.0 cluster on linux machines
client on windows 7
maven 3.0.4
maven repository http://repo1.maven.org/maven2/org/apache/hadoop/

• Assignee: Unassigned
• Reporter: Ran Levi
• Created: 19/Aug/14 15:00
• Updated: 20/Aug/14 12:27
• Resolved: 20/Aug/14 12:27

Description

When adding spark-yarn_2.10:1.0.2 dependency to java project, other hadoop-yarn-XXX dependencies are needed. Those dependencies are downloaded using version 1.0.4 which does not exist, resulting in build error.
Version 1.0.4 is taken from hadoop.version variable in spark-parent-1.0.2.pom.