Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.0.0
-
None
-
None
-
None
Description
This JIRA relates to the mail thread here:
http://mail-archives.apache.org/mod_mbox/incubator-giraph-dev/201203.mbox/browser
Currently we check for the munge flags HADOOP, HADOOP_FACEBOOK and HADOOP_NON_SECURE when using munge in a few places. Hopefully we can eliminate usage of munge in the future, but until then, we can mitigate the complexity by consolidating the number of flags checked. This JIRA renames HADOOP_FACEBOOK to HADOOP_SECURE, and removes usages of HADOOP, to handle the same conditional compilation requirements. It also makes it easier to add more maven profiles so that we can easily increase our hadoop version coverage.
This patch modifies the existing hadoop_facebook profile to use the new HADOOP_SECURE munge flag, rather than HADOOP_FACEBOOK.
It also adds a new hadoop maven profile, hadoop_trunk, which also sets HADOOP_SECURE.
Finally, it adds a default profile, hadoop_0.20.203. This is needed so that we can specify its dependencies separately from hadoop_trunk, because the hadoop dependencies have changed between trunk and 0.205.0 - the former requires hadoop-common, hadoop-mapreduce-client-core, and hadoop-mapreduce-client-common, whereas the latter requires hadoop-core.
With this patch, the following passes:
mvn clean verify && mvn -Phadoop_trunk clean verify && mvn -Phadoop_0.20.203 clean verify
Current problems:
- I left in place the usage of HADOOP_NON_SECURE, but note that the profile that uses this is hadoop_non_secure, which fails to compile on trunk: https://issues.apache.org/jira/browse/GIRAPH-167 .
- I couldn't get -Phadoop_facebook to work; does this work outside of Facebook?
Attachments
Attachments
Issue Links
- is related to
-
GIRAPH-212 Security is busted since GIRAPH-168
- Resolved
-
GIRAPH-183 Add Claudio's FOSDEM presentation (slides and video) to the site
- Resolved
-
GIRAPH-196 Get rid of the munge-maven-plugin
- Open
-
GIRAPH-485 Hadoop Shims, smart detect version on job start
- Open
- relates to
-
GIRAPH-14 Support for the Facebook Hadoop branch
- Resolved
-
HADOOP-6419 Change RPC layer to support SASL based mutual authentication
- Closed
-
HADOOP-6904 A baby step towards inter-version RPC communications
- Closed