[GIRAPH-959] Giraph's 1.1.0 hadoop_yarn profile can no longer be built with hadoop 2.0.3-alpha

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.1.0
• Component/s: build
• Labels: build dependencies hadoop-version
• Assignee: Unassigned
• Reporter: Philipp Nolte
• Created: 26/Oct/14 17:01
• Updated: 16/Dec/14 00:00

Description

Trying to build giraph release 1.1.0-RC1 for hadoop 2.0.3-alpha with profile hadoop_yarn

$ git clone git://git.apache.org/giraph.git
$ git checkout release-1.1.0-RC1
$ mvn -Dhadoop.version=2.0.3-alpha -DskipTests -Phadoop_yarn clean package

fails with lots of cannot find symbol errors:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project giraph-core: Compilation failure: Compilation failure:
[ERROR] /Users/philipp/Code/giraph/giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnClient.java:[49,41] package org.apache.hadoop.yarn.client.api does not exist
[ERROR] /Users/philipp/Code/giraph/giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnClient.java:[50,41] package org.apache.hadoop.yarn.client.api does not exist
[ERROR] /Users/philipp/Code/giraph/giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnClient.java:[52,41] cannot find symbol
[ERROR] symbol:   class YarnException
[ERROR] location: package org.apache.hadoop.yarn.exceptions
[ERROR] /Users/philipp/Code/giraph/giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnClient.java:[88,11] cannot find symbol
[ERROR] symbol:   class YarnClient
[ERROR] location: class org.apache.giraph.yarn.GiraphYarnClient
[ERROR] /Users/philipp/Code/giraph/giraph-core/target/munged/main/org/apache/giraph/yarn/GiraphYarnClient.java:[115,52] cannot find symbol

This is probably due to missing dependencies in the hadoop_yarn profile. It may also mean, that giraph's hadoop_yarn profile is no longer compatible with hadoop 2.0.3-alpha, as hadoop-yarn-project version 2.0.3-alpha does not include the package org.apache.hadoop.yarn.client.api for example.

In latter case, the pom.xml comment stating that hadoop_yarn runs on hadoop 2.0.3-alpha by default is deprecated and should be removed to prevent confusion.

What versions of hadoop can I build giraph version 1.1.0 with the hadoop_yarn profile with?