[MAPREDUCE-3367] Consolidate MRv2 maven modules

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.23.0
• Component/s: build, mrv2

Environment

Java, Maven

• Assignee: Eric Yang
• Reporter: Eric Yang
• Created: 07/Nov/11 17:20
• Updated: 07/Nov/11 17:50
• Resolved: 07/Nov/11 17:50

Description

Maven modules for MRv2 has been splitter into many small pieces, like this:

[INFO] hadoop-yarn ....................................... SUCCESS [0.312s]
[INFO] hadoop-yarn-api ................................... SUCCESS [24.184s]
[INFO] hadoop-yarn-common ................................ SUCCESS [18.243s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.094s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [5.394s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [7.056s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [2.295s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [10.217s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.395s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.069s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [2.534s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.144s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.073s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [13.727s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [13.045s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [1.909s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [6.676s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [3.317s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [2.575s]
[INFO] hadoop-mapreduce .................................. SUCCESS [4.952s]

The excessive use of directory structure is slowing down development and compile time. For example, hadoop-yarn-server-tests and hadoop-yarn-site should not be stand alone modules. There are better ways to organize the source code. The proposal is to rearrange the code to:

Apache Hadoop Mapreduce Project
  - Apache Hadoop Mapreduce Client
  - Apache Hadoop YARN

This is a nice to have for 0.23, but optional.

Issue Links

• duplicates: MAPREDUCE-2600 MR-279: simplify the jars