Support Maven-based builds

• Key: HADOOP-3302
• Type: New Feature
• Status: Open
• Priority: Major
• Assignee: Edward J. Yoon
• Reporter: Edward J. Yoon
• Affects Version/s: 0.18.0

Description

The reasons I would like to use maven are:

• the possibility to define artifact templates to define a kind of standard layout/design by artifact
• it is not necessary for every developer to come up with his own ant build-file and process
• the possibility to define and resolve dependencies transitively

But there are also some disadvantages/concerns I identified:
Maven is downloading a lot of plugins from a central repository that is not under my control

• What's about the licenses of these plugins? How do I know I am allowed to use them for a commercial product?
• What's about security? How can I be sure, that the plugins are not manipulated and contain the original (delivered by the JAR provider for e.g. junit-jar) contents. I observed, that some plugins didn't pass the md5 checks but have been installed anyway.

Comments

I am supportive of having a POM for the hadoop artifacts and publishing in the repository...I will file a related patch to add those

However, the other concerns here: library security, licensing, are not in scope for this project. They are an consequence of using Maven or Ivy.

for reference
1. artifacts are MD5 signed; this can detect accidental corruption but not malicious artifacts (including malicious metadata)
2. some POMS contain license information, but this is patchy; you are left to determine for yourself whether JARs are compatible.
3. Apache are fairly strict about preventing unapproved apache artifacts getting into the repository, but the process is vulneable to someone malicious uploading a third party JAR (such as jedit) which has been subverted

if these issues concern you, you shouldnt be using public repositories and transitive dependencies.

[Hiram Chirino]

1) While maven is more exposed to malicious repository attacks due to it automatically getting dependencies, that same attack can affect someone if they download dependencies manually. Hopefully maven one day will provide a better signature checking facility.
2) Agreed dependencies of the final distro need to be manually checked but this should be getting done regardless. This does not mean that build time dependencies all need to be ASL compatible, jut the run time and distribution artifacts.
3) Once again this one affects any repo, not just maven repo users.

So I'm hoping that artifact signature checking becomes an option in maven day.

[Hiram Chirino]

I've just created a new maven plugin that will resolve Steve's #1 and #3 objection. Source to the new maven plugin can be found here:
https://svn.apache.org/repos/asf/servicemix/maven-plugins/checksum-maven-plugin/trunk

Basically the plugin supports generating a checksum.txt file that is included as part of the project build. This file holds all the checksums for the dependencies (includes checksums for the metadata too) of the project.

It then validates the checksums of the downloaded dependencies against those stored in the checksum.txt file.

[Steve Loughran]

rolling back accidental deletion. Edited wrong bugrep