Hadoop Common
  1. Hadoop Common
  2. HADOOP-7412 Mavenization Umbrella
  3. HADOOP-7560

Make hadoop-common a POM module with sub-modules (common & alfredo)

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.

      Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.

      I.e.:

      mkdir hadoop-common-project

      mv hadoop-common hadoop-common-project

      mv hadoop-alfredo hadoop-common-project

      hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo

      1. HADOOP-7560v1.sh
        2 kB
        Alejandro Abdelnur
      2. HADOOP-7560v1.patch
        10 kB
        Alejandro Abdelnur
      3. HADOOP-7560v2.sh
        2 kB
        Alejandro Abdelnur
      4. HADOOP-7560v2.patch
        7 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          Alejandro Abdelnur added a comment -

          If this is done the same should be done for hadoop-hdfs, hadoop-mapreduce already does that as it had submodules at time of integration

          Show
          Alejandro Abdelnur added a comment - If this is done the same should be done for hadoop-hdfs, hadoop-mapreduce already does that as it had submodules at time of integration
          Hide
          Doug Cutting added a comment -

          This will require updating the svn mailer configuration again, right? I can help with that again.

          Show
          Doug Cutting added a comment - This will require updating the svn mailer configuration again, right? I can help with that again.
          Hide
          Arun C Murthy added a comment -

          So, is this the new way of doing contrib modules? Is alfredo meant to be one?

          Show
          Arun C Murthy added a comment - So, is this the new way of doing contrib modules? Is alfredo meant to be one?
          Hide
          Arun C Murthy added a comment -

          Also, is it necessary to move hadoop-common to hadoop-common-project?

          Show
          Arun C Murthy added a comment - Also, is it necessary to move hadoop-common to hadoop-common-project?
          Hide
          Alejandro Abdelnur added a comment -

          @Arun, hadoop-alfredo is a module used by hadoop-common. Until now contrib modules have been always the other way around (built on top of common/hdfs/mapreduce).

          I think we need to discuss a bit the general layout of things. IMO, MR2 module approach is good, shouldn't we do something similar for common & hdfs?

          Show
          Alejandro Abdelnur added a comment - @Arun, hadoop-alfredo is a module used by hadoop-common. Until now contrib modules have been always the other way around (built on top of common/hdfs/mapreduce). I think we need to discuss a bit the general layout of things. IMO, MR2 module approach is good, shouldn't we do something similar for common & hdfs?
          Hide
          Arun C Murthy added a comment -

          What was the idea when we committed alfredo? I want to ensure I'm not missing something in my understanding...

          Show
          Arun C Murthy added a comment - What was the idea when we committed alfredo? I want to ensure I'm not missing something in my understanding...
          Hide
          Alejandro Abdelnur added a comment -

          @Arun, I assume you are referring to the moving common/hdfs to submodules. If so, we just started the discussion here. I don't see any proposal/agreement yet.

          The current set of Maven modules is a follows (the '*' means that are aggregator Maven modules, type POM, no code in them):

          trunk/
                hadopo-project/
                hadoop-project-dist/
                hadoop-assemblies/
                hadoop-annotations/
                hadoop-alfredo/
                hadoop-common/
                hadoop-hdfs/
                hadoop-mapreduce/*
                                 hadoop-mr-client/*
                                                  hadoop-mapreduce-client-app/
                                                  hadoop-mapreduce-client-common/
                                                  hadoop-mapreduce-client-core/
                                                  hadoop-mapreduce-client-hs/
                                                  hadoop-mapreduce-client-jobclient/
                                                  hadoop-mapreduce-client-shuffle/                                       
                                 hadoop-yarn/*
                                             hadoop-yarn-api/
                                             hadoop-yarn-common/
                                             hadoop-yarn-server/
          

          We could do something like (adding hoop modules to HDFS to have more comprehensive examples). I'm using AG as a module label to indicate it is an aggregator and no-code/artifact is produced from that module:

          trunk/
                hadopo-project/
                hadoop-project-dist/
                hadoop-assemblies/
                hadoop-annotations/
                hadoop-common-AG/*
                              hadoop-alfredo/
                              hadoop-common/
                hadoop-hdfs-AG/*
                            hadoop-hdfs/
                            hadoop-hoop-client/
                            hadoop-hoop-server/
                hadoop-mapreduce-AG/*
                                 hadoop-mr-client-AG/
                                                  hadoop-mapreduce-client-app/
                                                  hadoop-mapreduce-client-common/
                                                  hadoop-mapreduce-client-core/
                                                  hadoop-mapreduce-client-hs/
                                                  hadoop-mapreduce-client-jobclient/
                                                  hadoop-mapreduce-client-shuffle/                                       
                                 hadoop-yarn-AG/
                                             hadoop-yarn-api/
                                             hadoop-yarn-common/
                                             hadoop-yarn-server/
          

          I've introduce AG because, IMO, it would be confusing to have hadoop-common at two levels. I've change all aggregator modules to AG for consistency.

          Thoughts?

          Show
          Alejandro Abdelnur added a comment - @Arun, I assume you are referring to the moving common/hdfs to submodules. If so, we just started the discussion here. I don't see any proposal/agreement yet. The current set of Maven modules is a follows (the '*' means that are aggregator Maven modules, type POM, no code in them): trunk/ hadopo-project/ hadoop-project-dist/ hadoop-assemblies/ hadoop-annotations/ hadoop-alfredo/ hadoop-common/ hadoop-hdfs/ hadoop-mapreduce/* hadoop-mr-client/* hadoop-mapreduce-client-app/ hadoop-mapreduce-client-common/ hadoop-mapreduce-client-core/ hadoop-mapreduce-client-hs/ hadoop-mapreduce-client-jobclient/ hadoop-mapreduce-client-shuffle/ hadoop-yarn/* hadoop-yarn-api/ hadoop-yarn-common/ hadoop-yarn-server/ We could do something like (adding hoop modules to HDFS to have more comprehensive examples). I'm using AG as a module label to indicate it is an aggregator and no-code/artifact is produced from that module: trunk/ hadopo-project/ hadoop-project-dist/ hadoop-assemblies/ hadoop-annotations/ hadoop-common-AG/* hadoop-alfredo/ hadoop-common/ hadoop-hdfs-AG/* hadoop-hdfs/ hadoop-hoop-client/ hadoop-hoop-server/ hadoop-mapreduce-AG/* hadoop-mr-client-AG/ hadoop-mapreduce-client-app/ hadoop-mapreduce-client-common/ hadoop-mapreduce-client-core/ hadoop-mapreduce-client-hs/ hadoop-mapreduce-client-jobclient/ hadoop-mapreduce-client-shuffle/ hadoop-yarn-AG/ hadoop-yarn-api/ hadoop-yarn-common/ hadoop-yarn-server/ I've introduce AG because, IMO, it would be confusing to have hadoop-common at two levels. I've change all aggregator modules to AG for consistency. Thoughts?
          Hide
          Eric Yang added a comment -

          Why don't we express hadoop-common depends on hadoop-alfredo and this will put hadoop-alfredo into share/hadoop/common/lib. This approach would be same as how hadoop-common includes Log4J jar files. There is no need of aggregator modules.

          Show
          Eric Yang added a comment - Why don't we express hadoop-common depends on hadoop-alfredo and this will put hadoop-alfredo into share/hadoop/common/lib. This approach would be same as how hadoop-common includes Log4J jar files. There is no need of aggregator modules.
          Hide
          Doug Cutting added a comment -

          As noted in HDFS-2178, hadoop-alfredo should be named hadoop-auth, and hadoop-hoop should be named hadoop-hdfs-http.

          Show
          Doug Cutting added a comment - As noted in HDFS-2178 , hadoop-alfredo should be named hadoop-auth, and hadoop-hoop should be named hadoop-hdfs-http.
          Hide
          Luke Lu added a comment -

          I've introduce AG because, IMO, it would be confusing to have hadoop-common at two levels. I've change all aggregator modules to AG for consistency.

          How about use -modules for aggregation, e.g. hadoop-common-modules as this doesn't require further explanation.

          Why don't we express hadoop-common depends on hadoop-alfredo...

          The idea is to include the source into the hadoop common modules. The modules is the new (IMO better) way to include contrib sources.

          Show
          Luke Lu added a comment - I've introduce AG because, IMO, it would be confusing to have hadoop-common at two levels. I've change all aggregator modules to AG for consistency. How about use -modules for aggregation, e.g. hadoop-common-modules as this doesn't require further explanation. Why don't we express hadoop-common depends on hadoop-alfredo... The idea is to include the source into the hadoop common modules. The modules is the new (IMO better) way to include contrib sources.
          Hide
          Arun C Murthy added a comment -

          I'm not sure another huge set of 'svn mv's to incorporate alfredo is in order.

          If hadoop-common depends on hadoop-alfredo, why don't we just move alfredo into common and be done with?

          The common jar isn't that big really, so anyone who depends on hadoop-alfredo could just depend on hadoop-common?

          Show
          Arun C Murthy added a comment - I'm not sure another huge set of 'svn mv's to incorporate alfredo is in order. If hadoop-common depends on hadoop-alfredo, why don't we just move alfredo into common and be done with? The common jar isn't that big really, so anyone who depends on hadoop-alfredo could just depend on hadoop-common?
          Hide
          Arun C Murthy added a comment -

          I'm not particularly disposed either way, I'd like this resolved prior to the 0.23 branch if possible.

          This and MAPREDUCE-2838 are the only things blocking it IMO.

          Show
          Arun C Murthy added a comment - I'm not particularly disposed either way, I'd like this resolved prior to the 0.23 branch if possible. This and MAPREDUCE-2838 are the only things blocking it IMO.
          Hide
          Alejandro Abdelnur added a comment -

          Arun, I'd like to keep Alfredo as a separate module. The main reason is that that its runtime dependencies are minimal:

          [INFO] org.apache.hadoop:hadoop-alfredo:jar:0.23.0-SNAPSHOT
          [INFO] +- org.slf4j:slf4j-api:jar:1.5.11:compile
          [INFO] +- commons-codec:commons-codec:jar:1.4:compile
          [INFO] +- log4j:log4j:jar:1.2.15:compile
          [INFO] \- org.slf4j:slf4j-log4j12:jar:1.5.11:compile
          

          (plus slf4j & log4j don't come when used in client mode, they are optional)

          On the other hand, Hadoop Common dependencies are:

          [INFO] org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT
          [INFO] +- com.google.guava:guava:jar:r09:compile
          [INFO] +- commons-cli:commons-cli:jar:1.2:compile
          [INFO] +- org.apache.commons:commons-math:jar:2.1:compile
          [INFO] +- xmlenc:xmlenc:jar:0.52:compile
          [INFO] +- commons-httpclient:commons-httpclient:jar:3.1:compile
          [INFO] +- commons-codec:commons-codec:jar:1.4:compile
          [INFO] +- commons-net:commons-net:jar:1.4.1:compile
          [INFO] +- javax.servlet:servlet-api:jar:2.5:compile
          [INFO] +- org.mortbay.jetty:jetty:jar:6.1.26:compile
          [INFO] +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
          [INFO] +- tomcat:jasper-compiler:jar:5.5.23:compile
          [INFO] +- tomcat:jasper-runtime:jar:5.5.23:compile
          [INFO] +- javax.servlet.jsp:jsp-api:jar:2.1:compile
          [INFO] +- commons-el:commons-el:jar:1.0:compile
          [INFO] +- commons-logging:commons-logging:jar:1.1.1:compile
          [INFO] +- commons-logging:commons-logging-api:jar:1.1:compile
          [INFO] +- log4j:log4j:jar:1.2.15:compile
          [INFO] +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
          [INFO] +- commons-lang:commons-lang:jar:2.5:compile
          [INFO] +- commons-collections:commons-collections:jar:3.2.1:compile
          [INFO] +- commons-configuration:commons-configuration:jar:1.6:compile
          [INFO] |  +- commons-digester:commons-digester:jar:1.8:compile
          [INFO] |  |  \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
          [INFO] |  \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
          [INFO] +- hsqldb:hsqldb:jar:1.8.0.7:compile
          [INFO] +- org.slf4j:slf4j-api:jar:1.5.11:compile
          [INFO] +- org.slf4j:slf4j-log4j12:jar:1.5.11:compile
          [INFO] +- org.eclipse.jdt:core:jar:3.1.1:compile
          [INFO] +- oro:oro:jar:2.0.8:compile
          [INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.6.9:compile
          [INFO] |  \- org.codehaus.jackson:jackson-core-asl:jar:1.6.9:compile
          [INFO] +- org.aspectj:aspectjrt:jar:1.6.5:compile
          [INFO] +- org.apache.avro:avro:jar:1.5.2:compile
          [INFO] |  +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
          [INFO] |  \- org.xerial.snappy:snappy-java:jar:1.0.3-rc4:compile
          [INFO] +- org.apache.avro:avro-ipc:jar:1.5.2:compile
          [INFO] |  +- org.jboss.netty:netty:jar:3.2.4.Final:compile
          [INFO] |  \- org.apache.velocity:velocity:jar:1.7:compile
          [INFO] +- net.sf.kosmosfs:kfs:jar:0.3:compile
          [INFO] +- com.google.protobuf:protobuf-java:jar:2.4.0a:compile
          [INFO] \- org.apache.hadoop:hadoop-alfredo:jar:0.23.0-SNAPSHOT:compile
          

          This is not really minimal. And having to exclude all of them when just wanting to use Alfredo client it not very developer friendly.

          Regarding the module names and organization, I'll be posting a comment a bit later today once I finish writing it up.

          Show
          Alejandro Abdelnur added a comment - Arun, I'd like to keep Alfredo as a separate module. The main reason is that that its runtime dependencies are minimal: [INFO] org.apache.hadoop:hadoop-alfredo:jar:0.23.0-SNAPSHOT [INFO] +- org.slf4j:slf4j-api:jar:1.5.11:compile [INFO] +- commons-codec:commons-codec:jar:1.4:compile [INFO] +- log4j:log4j:jar:1.2.15:compile [INFO] \- org.slf4j:slf4j-log4j12:jar:1.5.11:compile (plus slf4j & log4j don't come when used in client mode, they are optional) On the other hand, Hadoop Common dependencies are: [INFO] org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT [INFO] +- com.google.guava:guava:jar:r09:compile [INFO] +- commons-cli:commons-cli:jar:1.2:compile [INFO] +- org.apache.commons:commons-math:jar:2.1:compile [INFO] +- xmlenc:xmlenc:jar:0.52:compile [INFO] +- commons-httpclient:commons-httpclient:jar:3.1:compile [INFO] +- commons-codec:commons-codec:jar:1.4:compile [INFO] +- commons-net:commons-net:jar:1.4.1:compile [INFO] +- javax.servlet:servlet-api:jar:2.5:compile [INFO] +- org.mortbay.jetty:jetty:jar:6.1.26:compile [INFO] +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile [INFO] +- tomcat:jasper-compiler:jar:5.5.23:compile [INFO] +- tomcat:jasper-runtime:jar:5.5.23:compile [INFO] +- javax.servlet.jsp:jsp-api:jar:2.1:compile [INFO] +- commons-el:commons-el:jar:1.0:compile [INFO] +- commons-logging:commons-logging:jar:1.1.1:compile [INFO] +- commons-logging:commons-logging-api:jar:1.1:compile [INFO] +- log4j:log4j:jar:1.2.15:compile [INFO] +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile [INFO] +- commons-lang:commons-lang:jar:2.5:compile [INFO] +- commons-collections:commons-collections:jar:3.2.1:compile [INFO] +- commons-configuration:commons-configuration:jar:1.6:compile [INFO] | +- commons-digester:commons-digester:jar:1.8:compile [INFO] | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile [INFO] | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile [INFO] +- hsqldb:hsqldb:jar:1.8.0.7:compile [INFO] +- org.slf4j:slf4j-api:jar:1.5.11:compile [INFO] +- org.slf4j:slf4j-log4j12:jar:1.5.11:compile [INFO] +- org.eclipse.jdt:core:jar:3.1.1:compile [INFO] +- oro:oro:jar:2.0.8:compile [INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.6.9:compile [INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.6.9:compile [INFO] +- org.aspectj:aspectjrt:jar:1.6.5:compile [INFO] +- org.apache.avro:avro:jar:1.5.2:compile [INFO] | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile [INFO] | \- org.xerial.snappy:snappy-java:jar:1.0.3-rc4:compile [INFO] +- org.apache.avro:avro-ipc:jar:1.5.2:compile [INFO] | +- org.jboss.netty:netty:jar:3.2.4.Final:compile [INFO] | \- org.apache.velocity:velocity:jar:1.7:compile [INFO] +- net.sf.kosmosfs:kfs:jar:0.3:compile [INFO] +- com.google.protobuf:protobuf-java:jar:2.4.0a:compile [INFO] \- org.apache.hadoop:hadoop-alfredo:jar:0.23.0-SNAPSHOT:compile This is not really minimal. And having to exclude all of them when just wanting to use Alfredo client it not very developer friendly. Regarding the module names and organization, I'll be posting a comment a bit later today once I finish writing it up.
          Hide
          Arun C Murthy added a comment -

          Alejandro - is there a way we can effect this without another 'svn mv' and renaming everything under hadoop/trunk?

          Show
          Arun C Murthy added a comment - Alejandro - is there a way we can effect this without another 'svn mv' and renaming everything under hadoop/trunk?
          Hide
          Eric Yang added a comment -

          I think making hadoop-common aggregator module would be a recursive mistake. It should be either make alfredo part of hadoop-common, or hadoop-common depends on hadoop-auth (which is the alfredo module). I am more in favor of making hadoop-http, hadoop-auth part of hadoop-common module.

          Show
          Eric Yang added a comment - I think making hadoop-common aggregator module would be a recursive mistake. It should be either make alfredo part of hadoop-common, or hadoop-common depends on hadoop-auth (which is the alfredo module). I am more in favor of making hadoop-http, hadoop-auth part of hadoop-common module.
          Hide
          Alejandro Abdelnur added a comment -

          @Arun, I'm afraid we'd have to bite another 'svn mv'

          @Eric, IMO hadoop-alfredo, to be hadoop-auth, should be a separate module from hadoop-common (per reasons in my previous comment). If hadoop-common becomes an aggregator module or not it depends on how do we decide to structure Maven modules, hierarchical (like MR2) or flat (like hadoop-common, hadoop-hdfs, hadoop-alfredo, hadoop-project, ...).

          Working on a comment for the last stuff, coming soon.

          Show
          Alejandro Abdelnur added a comment - @Arun, I'm afraid we'd have to bite another 'svn mv' @Eric, IMO hadoop-alfredo, to be hadoop-auth, should be a separate module from hadoop-common (per reasons in my previous comment). If hadoop-common becomes an aggregator module or not it depends on how do we decide to structure Maven modules, hierarchical (like MR2) or flat (like hadoop-common, hadoop-hdfs, hadoop-alfredo, hadoop-project, ...). Working on a comment for the last stuff, coming soon.
          Hide
          Giridharan Kesavan added a comment -

          svn move again?

          since the beginning of this month I couldn't get a stable build out of trunk; Every week we have some new changes to make to the build env and something is broken. It's getting very painful.

          Show
          Giridharan Kesavan added a comment - svn move again? since the beginning of this month I couldn't get a stable build out of trunk; Every week we have some new changes to make to the build env and something is broken. It's getting very painful.
          Hide
          Arun C Murthy added a comment -

          How about adding a top-level hadoop-extras and putting hadoop-alfredo there?

          Show
          Arun C Murthy added a comment - How about adding a top-level hadoop-extras and putting hadoop-alfredo there?
          Hide
          Luke Lu added a comment -

          How about adding a top-level hadoop-extras and putting hadoop-alfredo there?

          This is a good idea to minimize svn mv . Though I'd like to avoid "extras", as it can imply something similar to apache-extras, which is not apache licensed. IMO, hadoop-optional-modules would be a better name.

          Show
          Luke Lu added a comment - How about adding a top-level hadoop-extras and putting hadoop-alfredo there? This is a good idea to minimize svn mv . Though I'd like to avoid "extras", as it can imply something similar to apache-extras, which is not apache licensed. IMO, hadoop-optional-modules would be a better name.
          Hide
          Mahadev konar added a comment -

          How about adding a top-level hadoop-extras and putting hadoop-alfredo there?

          +1. I like this idea. I am also not in favor of another set of svn moves. Its very disrupting.

          Show
          Mahadev konar added a comment - How about adding a top-level hadoop-extras and putting hadoop-alfredo there? +1. I like this idea. I am also not in favor of another set of svn moves. Its very disrupting.
          Hide
          Alejandro Abdelnur added a comment -

          I'm trying to take as step back here to see the general issue of sub-modules rather than the pinpoint solution for alfredo or hoop.

          Comment quite long, two main sections: Flat vs Hierarchical module structure & dual artifacts modules.

          Flat vs Hierarchical module structure

          For the purposes of the discussion, I'm leaving aside the concerns of another SVN move.

          When organizing a multi-module Maven project there are 2 approaches, using a flat module structure or using a hierarchical module structure.

          Our current layout is a mix of both (for the purposes of the generalization, lets assume hoop is in as well):

          trunk/
                hadopo-project/ (f)
                hadoop-project-dist/ (f)
                hadoop-assemblies/ (f)
                hadoop-annotations/ (f)
                hadoop-auth/ (f) (this is alfredo)
                hadoop-common/ (f)
                hadoop-hdfs/ (f)
                hadoop-hdfs-http-client/ (f) (this is hoop)
                hadoop-hdfs-http-server/ (f) (this is hoop)
                hadoop-mapreduce/ (h) (*)
                                 hadoop-mr-client/ (*)
                                                  hadoop-mapreduce-client-app/
                                                  hadoop-mapreduce-client-common/
                                                  hadoop-mapreduce-client-core/
                                                  hadoop-mapreduce-client-hs/
                                                  hadoop-mapreduce-client-jobclient/
                                                  hadoop-mapreduce-client-shuffle/                                       
                                 hadoop-yarn/ (*)
                                             hadoop-yarn-api/
                                             hadoop-yarn-common/
                                             hadoop-yarn-server/
          

          (f: flat modules, h: hierarchical modules, *: aggregator modules)

          1. Note that the module names, flat or hierarchical, denote the hierarchy. Those names also denote the artifact name produced the module (typically a JAR). Because of this I think module names should stay as they are regardless.

          2. The nice thing about a hierarchal structure is that we use directory levels to naturally represent the hierarchy. Still, because of #1, we shouldn't simplify the names. The bad thing, is that navigating through directory paths becomes a bit more cumbersome/long. Another issue is that we need additional aggregator modules (like hadoop-mapreduce, hadoop-mr-client and hadoop-yarn).

          3. The nice thing about flat structure is that directory navigation is easy, the structure is in the names already. The bad thing is that the number of directories at trunk/ level is larger. The bad thing is that the sense of module scope is lost. IDEs like IntelliJ and Eclipse flatten the hierarchy. Thus, a flatten structure could be a bit more intuitive.

          4. Maven doesn't care if flat or hierarchical, it resolves the order of building based on dependencies.

          Personally, I'd prefer a flat structure, but I'm OK either way. What I would definitely like is to have full consistency.

          Dual Artifact Modules

          Currently hadoop-common and hadoop-hdfs modules produce 2 types of artifacts, a JAR and a TAR.

          In my opinion, a Maven module should produce a single artifact, a JAR or a TAR. This is, for Common and HDFS we should have a hadoop-common-dist and a hadoop-hdfs-dist modules that only do assembly work. This decoupling will easily allow to bundle hoop with HDFS.

          Otherwise, if Hoop depends on HDFS (it must for testing) there is a circular dependency (Hoop depends on HDFS for build/testing, HDFS depends on Hoop for packaging).

          Because of this I'd advocate for single artifact modules.

          Show
          Alejandro Abdelnur added a comment - I'm trying to take as step back here to see the general issue of sub-modules rather than the pinpoint solution for alfredo or hoop. Comment quite long, two main sections: Flat vs Hierarchical module structure & dual artifacts modules. Flat vs Hierarchical module structure For the purposes of the discussion, I'm leaving aside the concerns of another SVN move. When organizing a multi-module Maven project there are 2 approaches, using a flat module structure or using a hierarchical module structure. Our current layout is a mix of both (for the purposes of the generalization, lets assume hoop is in as well): trunk/ hadopo-project/ (f) hadoop-project-dist/ (f) hadoop-assemblies/ (f) hadoop-annotations/ (f) hadoop-auth/ (f) ( this is alfredo) hadoop-common/ (f) hadoop-hdfs/ (f) hadoop-hdfs-http-client/ (f) ( this is hoop) hadoop-hdfs-http-server/ (f) ( this is hoop) hadoop-mapreduce/ (h) (*) hadoop-mr-client/ (*) hadoop-mapreduce-client-app/ hadoop-mapreduce-client-common/ hadoop-mapreduce-client-core/ hadoop-mapreduce-client-hs/ hadoop-mapreduce-client-jobclient/ hadoop-mapreduce-client-shuffle/ hadoop-yarn/ (*) hadoop-yarn-api/ hadoop-yarn-common/ hadoop-yarn-server/ (f: flat modules, h: hierarchical modules, *: aggregator modules) 1. Note that the module names, flat or hierarchical, denote the hierarchy. Those names also denote the artifact name produced the module (typically a JAR). Because of this I think module names should stay as they are regardless. 2. The nice thing about a hierarchal structure is that we use directory levels to naturally represent the hierarchy. Still, because of #1, we shouldn't simplify the names. The bad thing, is that navigating through directory paths becomes a bit more cumbersome/long. Another issue is that we need additional aggregator modules (like hadoop-mapreduce, hadoop-mr-client and hadoop-yarn). 3. The nice thing about flat structure is that directory navigation is easy, the structure is in the names already. The bad thing is that the number of directories at trunk/ level is larger. The bad thing is that the sense of module scope is lost. IDEs like IntelliJ and Eclipse flatten the hierarchy. Thus, a flatten structure could be a bit more intuitive. 4. Maven doesn't care if flat or hierarchical, it resolves the order of building based on dependencies. Personally, I'd prefer a flat structure, but I'm OK either way. What I would definitely like is to have full consistency. Dual Artifact Modules Currently hadoop-common and hadoop-hdfs modules produce 2 types of artifacts, a JAR and a TAR. In my opinion, a Maven module should produce a single artifact, a JAR or a TAR. This is, for Common and HDFS we should have a hadoop-common-dist and a hadoop-hdfs-dist modules that only do assembly work. This decoupling will easily allow to bundle hoop with HDFS. Otherwise, if Hoop depends on HDFS (it must for testing) there is a circular dependency (Hoop depends on HDFS for build/testing, HDFS depends on Hoop for packaging). Because of this I'd advocate for single artifact modules.
          Hide
          Arun C Murthy added a comment -

          Alejandro - thanks for the write-up.

          I have a question - what were the approaches when we a) mavenized hadoop-common b) committed alfredo to trunk?

          I'm more concerned about ensuring we stabilize trunk so that normal builds etc. can be resumed - it's unfair on folks like Giri to keep changing the rules, he has to continuously scramble to keep up... hence, I'd rather avoid a large change to directory/package structure. We can revisit this at a later point (for hadoop-0.24 etc.) if necessary.

          Show
          Arun C Murthy added a comment - Alejandro - thanks for the write-up. I have a question - what were the approaches when we a) mavenized hadoop-common b) committed alfredo to trunk? I'm more concerned about ensuring we stabilize trunk so that normal builds etc. can be resumed - it's unfair on folks like Giri to keep changing the rules, he has to continuously scramble to keep up... hence, I'd rather avoid a large change to directory/package structure. We can revisit this at a later point (for hadoop-0.24 etc.) if necessary.
          Hide
          Alejandro Abdelnur added a comment -

          As discussed over IRC:

          • 1. we get the module structure correct in trunk
          • 2. we get trunk to build/run testcases
          • 3. we branch 0.23
          • 4. on 0.23 we stabilize the build/packaging
          • 5. on trunk we work on code
          • 6. once #4 is done, we port build from 0.23 to trunk
          • 7. we merge code changes from trunk to 0.23

          It will take a few days to do this but we won't block build work 0.23 & devel work on trunk after we reach #3

          We are shooting for immediate #1, #2 and #3.

          Straw-man patch will be avail shortly.

          Show
          Alejandro Abdelnur added a comment - As discussed over IRC: 1. we get the module structure correct in trunk 2. we get trunk to build/run testcases 3. we branch 0.23 4. on 0.23 we stabilize the build/packaging 5. on trunk we work on code 6. once #4 is done, we port build from 0.23 to trunk 7. we merge code changes from trunk to 0.23 It will take a few days to do this but we won't block build work 0.23 & devel work on trunk after we reach #3 We are shooting for immediate #1, #2 and #3. Straw-man patch will be avail shortly.
          Hide
          Arun C Murthy added a comment -

          +1, thanks for the effort on this Alejandro!

          Show
          Arun C Murthy added a comment - +1, thanks for the effort on this Alejandro!
          Hide
          Alejandro Abdelnur added a comment -

          'v1' script & patch.

          Flattens modules structure.

          To apply, from trunk/ dir do:

          $ sh HADOOP-7560v1.sh fs|svn
          

          (use 'fs' if working from GIT, use 'svn' if working from SVN)

          Then apply the patch.

          I did minimal changes in POMs to compile.

          Tested done:

          $ rm -rf ~/.m2/repositories/org/apache/hadoop/*
          $ mvn clean test -P-cbuild -DskipTests
          

          This ensures all Java dependencies/code is in place.

          [INFO] Reactor Summary:
          [INFO] 
          [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.664s]
          [INFO] Apache Hadoop Annotations ......................... SUCCESS [1.885s]
          [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [0.002s]
          [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.309s]
          [INFO] Apache Hadoop Auth ................................ SUCCESS [1.582s]
          [INFO] Apache Hadoop Common .............................. SUCCESS [17.927s]
          [INFO] Apache Hadoop HDFS ................................ SUCCESS [10.260s]
          [INFO] hadoop-mapreduce .................................. SUCCESS [0.064s]
          [INFO] hadoop-mapreduce-client ........................... SUCCESS [0.004s]
          [INFO] hadoop-yarn-api ................................... SUCCESS [4.713s]
          [INFO] hadoop-yarn-common ................................ SUCCESS [5.545s]
          [INFO] hadoop-mapreduce-client-core ...................... SUCCESS [2.693s]
          [INFO] hadoop-mapreduce-client-common .................... SUCCESS [2.444s]
          [INFO] hadoop-yarn-server-common ......................... SUCCESS [1.230s]
          [INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [1.768s]
          [INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [1.907s]
          [INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [0.424s]
          [INFO] hadoop-mapreduce-client-app ....................... SUCCESS [4.227s]
          [INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [0.838s]
          [INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.518s]
          [INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [1.063s]
          [INFO] hadoop-yarn ....................................... SUCCESS [0.003s]
          [INFO] hadoop-yarn-server ................................ SUCCESS [0.003s]
          [INFO] Apache Hadoop Main ................................ SUCCESS [0.005s]
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 1:01.912s
          
          Show
          Alejandro Abdelnur added a comment - 'v1' script & patch. Flattens modules structure. To apply, from trunk/ dir do: $ sh HADOOP-7560v1.sh fs|svn (use 'fs' if working from GIT, use 'svn' if working from SVN) Then apply the patch. I did minimal changes in POMs to compile. Tested done: $ rm -rf ~/.m2/repositories/org/apache/hadoop/* $ mvn clean test -P-cbuild -DskipTests This ensures all Java dependencies/code is in place. [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Project POM ......................... SUCCESS [1.664s] [INFO] Apache Hadoop Annotations ......................... SUCCESS [1.885s] [INFO] Apache Hadoop Project Dist POM .................... SUCCESS [0.002s] [INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.309s] [INFO] Apache Hadoop Auth ................................ SUCCESS [1.582s] [INFO] Apache Hadoop Common .............................. SUCCESS [17.927s] [INFO] Apache Hadoop HDFS ................................ SUCCESS [10.260s] [INFO] hadoop-mapreduce .................................. SUCCESS [0.064s] [INFO] hadoop-mapreduce-client ........................... SUCCESS [0.004s] [INFO] hadoop-yarn-api ................................... SUCCESS [4.713s] [INFO] hadoop-yarn-common ................................ SUCCESS [5.545s] [INFO] hadoop-mapreduce-client-core ...................... SUCCESS [2.693s] [INFO] hadoop-mapreduce-client-common .................... SUCCESS [2.444s] [INFO] hadoop-yarn-server-common ......................... SUCCESS [1.230s] [INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [1.768s] [INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [1.907s] [INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [0.424s] [INFO] hadoop-mapreduce-client-app ....................... SUCCESS [4.227s] [INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [0.838s] [INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.518s] [INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [1.063s] [INFO] hadoop-yarn ....................................... SUCCESS [0.003s] [INFO] hadoop-yarn-server ................................ SUCCESS [0.003s] [INFO] Apache Hadoop Main ................................ SUCCESS [0.005s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1:01.912s
          Hide
          Alejandro Abdelnur added a comment -

          Note that for now the hadoop-mapreduce, hadoop-mapreduce-client and hadoop-yarn modules (which where the old aggregator modules) stay becuase there POMs are being overloaded with other uses (dependencies and properties settings).

          A as part of a later consolidation of configuration they would go, but as they don't have code, they would go.

          In addition, MR1 test JAR stuff is under hadoop-mapreduce still

          Show
          Alejandro Abdelnur added a comment - Note that for now the hadoop-mapreduce, hadoop-mapreduce-client and hadoop-yarn modules (which where the old aggregator modules) stay becuase there POMs are being overloaded with other uses (dependencies and properties settings). A as part of a later consolidation of configuration they would go, but as they don't have code, they would go. In addition, MR1 test JAR stuff is under hadoop-mapreduce still
          Hide
          Arun C Murthy added a comment -

          Oh, I was under the impression that we were going to the heirarchical model, moving everything (e.g. hadoop-mr-client*) to the top-level really doesn't make sense to me.

          Show
          Arun C Murthy added a comment - Oh, I was under the impression that we were going to the heirarchical model, moving everything (e.g. hadoop-mr-client*) to the top-level really doesn't make sense to me.
          Hide
          Alejandro Abdelnur added a comment -

          from IRC

          [12:12pm] tucu: mmmhh, thought the other why around as my arguments in the JIRA to go flat were not disputed.
          [12:14pm] acmurthy: sorry
          [12:14pm] acmurthy: but having hadoop-yarn and hadoop-mr-client at top level seems very odd
          [12:14pm] acmurthy: miscommunication
          [12:14pm] tucu: still, logically still it is hierarchical, the names imply that. and IDEs (eclipse and idea) show modules flat, regardless of the hierachy. And Maven resolves the dependencies correctly as the hierachy is nothing for it
          [12:19pm] acmurthy: tucu - is it hard to do it fullly heirarchical?
          [12:19pm] acmurthy: having too many things at the top-level seems confusing for folks wihtout IDEs
          [12:19pm] acmurthy: i liked your prev. proposal (aggregator modules) much better
          [12:21pm] tucu: it is not hard, but i'd argue it is easier to work on a flat structure, ie to move file around, you'll always be at the same depth (*/src/main/java/). If you see big projects using maven the tent to use a flat set of modules.
          [12:21pm] tucu: and in the past, maven was not fully functional with multi-level stuff
          [12:22pm] tucu: now it is not the case, but some plugins may go bonkers
          [12:22pm] tucu: and the fact that IDEs show thigns flat, is easy for a devel to do things from both environments
          [12:22pm] tucu: without getting lost
          [12:25pm] acmurthy: i'd really prefer heirarchical
          [12:25pm] acmurthy: the project is truly heirarchical: common, hdfs, mr
          [12:25pm] acmurthy: and so n
          [12:25pm] acmurthy: on*
          [12:28pm] tucu: arun, let me work on a patch for hierachical approach and then we can discuss the merits of both
          
          Show
          Alejandro Abdelnur added a comment - from IRC [12:12pm] tucu: mmmhh, thought the other why around as my arguments in the JIRA to go flat were not disputed. [12:14pm] acmurthy: sorry [12:14pm] acmurthy: but having hadoop-yarn and hadoop-mr-client at top level seems very odd [12:14pm] acmurthy: miscommunication [12:14pm] tucu: still, logically still it is hierarchical, the names imply that. and IDEs (eclipse and idea) show modules flat, regardless of the hierachy. And Maven resolves the dependencies correctly as the hierachy is nothing for it [12:19pm] acmurthy: tucu - is it hard to do it fullly heirarchical? [12:19pm] acmurthy: having too many things at the top-level seems confusing for folks wihtout IDEs [12:19pm] acmurthy: i liked your prev. proposal (aggregator modules) much better [12:21pm] tucu: it is not hard, but i'd argue it is easier to work on a flat structure, ie to move file around, you'll always be at the same depth (*/src/main/java/). If you see big projects using maven the tent to use a flat set of modules. [12:21pm] tucu: and in the past, maven was not fully functional with multi-level stuff [12:22pm] tucu: now it is not the case , but some plugins may go bonkers [12:22pm] tucu: and the fact that IDEs show thigns flat, is easy for a devel to do things from both environments [12:22pm] tucu: without getting lost [12:25pm] acmurthy: i'd really prefer heirarchical [12:25pm] acmurthy: the project is truly heirarchical: common, hdfs, mr [12:25pm] acmurthy: and so n [12:25pm] acmurthy: on* [12:28pm] tucu: arun, let me work on a patch for hierachical approach and then we can discuss the merits of both
          Hide
          Eric Yang added a comment -

          Alejandro, the proposed approaches are two extreme models. How about we have a sensible approach to group top level modules and submodules?

          For example:

          trunk/
                hadopo-project/ (f)
                hadoop-common/ (f)
                hadoop-hdfs/ (h)
                                 hadoop-hdfs-http-client/ (f) (this is hoop)
                                 hadoop-hdfs-http-server/ (f) (this is hoop)
                hadoop-mapreduce/ (h) (*)
                                 hadoop-mr-client/ (*)
                                                  hadoop-mapreduce-client-app/
                                                  hadoop-mapreduce-client-common/
                                                  hadoop-mapreduce-client-core/
                                                  hadoop-mapreduce-client-hs/
                                                  hadoop-mapreduce-client-jobclient/
                                                  hadoop-mapreduce-client-shuffle/                                       
                                 hadoop-yarn/ (*)
                                             hadoop-yarn-api/
                                             hadoop-yarn-common/
                                             hadoop-yarn-server/
          

          In the proposed format, the structure of the tree is driven by the context and importance of the module.

          IMHO, hadoop-project-dist, hadoop-assemblies, hadoop-annotations should not be modules. They are small plugins and can be grouped as hadoop-project. hadoop-auth should be part of hadoop-common. This will remove the noise for the project organization.

          Show
          Eric Yang added a comment - Alejandro, the proposed approaches are two extreme models. How about we have a sensible approach to group top level modules and submodules? For example: trunk/ hadopo-project/ (f) hadoop-common/ (f) hadoop-hdfs/ (h) hadoop-hdfs-http-client/ (f) (this is hoop) hadoop-hdfs-http-server/ (f) (this is hoop) hadoop-mapreduce/ (h) (*) hadoop-mr-client/ (*) hadoop-mapreduce-client-app/ hadoop-mapreduce-client-common/ hadoop-mapreduce-client-core/ hadoop-mapreduce-client-hs/ hadoop-mapreduce-client-jobclient/ hadoop-mapreduce-client-shuffle/ hadoop-yarn/ (*) hadoop-yarn-api/ hadoop-yarn-common/ hadoop-yarn-server/ In the proposed format, the structure of the tree is driven by the context and importance of the module. IMHO, hadoop-project-dist, hadoop-assemblies, hadoop-annotations should not be modules. They are small plugins and can be grouped as hadoop-project. hadoop-auth should be part of hadoop-common. This will remove the noise for the project organization.
          Hide
          Doug Cutting added a comment -

          Eric, a non-Maven savvy observer, using both dashes and slashes to represent the hierarchy is redundant. If the module names themselves fully represent the hierarchy, what value does the directory structure add?

          Show
          Doug Cutting added a comment - Eric, a non-Maven savvy observer, using both dashes and slashes to represent the hierarchy is redundant. If the module names themselves fully represent the hierarchy, what value does the directory structure add?
          Hide
          Alejandro Abdelnur added a comment -

          Eric, your example (I guess c&p from a previous one of mine) lacks the aggregation level for common and hdfs.

          I'm finalizing a hierarchical version patch so we can compare things side to side.

          Show
          Alejandro Abdelnur added a comment - Eric, your example (I guess c&p from a previous one of mine) lacks the aggregation level for common and hdfs. I'm finalizing a hierarchical version patch so we can compare things side to side.
          Hide
          Arun C Murthy added a comment -
          Show
          Arun C Murthy added a comment - Thanks Alejandro - I'm looking fwd to it, your first proposal ( https://issues.apache.org/jira/browse/HADOOP-7560?focusedCommentId=13089808&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13089808 ) seems reasonable to me.
          Hide
          Giridharan Kesavan added a comment -

          +1 to Eric's proposal.
          Not everyone would want to checkout the whole trunk to just build mapreduce or hdfs or yarn.
          With the Hierarchical approach we get the liberty of just checking out hadoop-hdfs or hadoop-mapreduce or hadoop-yarn and build it without having to checkout the whole trunk. And, anyways maven would take care of the dependency resolutions.

          Show
          Giridharan Kesavan added a comment - +1 to Eric's proposal. Not everyone would want to checkout the whole trunk to just build mapreduce or hdfs or yarn. With the Hierarchical approach we get the liberty of just checking out hadoop-hdfs or hadoop-mapreduce or hadoop-yarn and build it without having to checkout the whole trunk. And, anyways maven would take care of the dependency resolutions.
          Hide
          Alejandro Abdelnur added a comment -

          Just for the sake of the discussion, Yarn should not be under mapreduce but next to it, right?

          Show
          Alejandro Abdelnur added a comment - Just for the sake of the discussion, Yarn should not be under mapreduce but next to it, right?
          Hide
          Alejandro Abdelnur added a comment -

          v2 script/patch (same instructions) set a hierarchical layout.

          Show
          Alejandro Abdelnur added a comment - v2 script/patch (same instructions) set a hierarchical layout.
          Hide
          Arun C Murthy added a comment -

          Alejandro, I think Giri's comment makes sense - for now having hadoop-yarn in hadoop-mapreduce helps folks looking for a single checkout... so, we could keep it there. Thoughts?

          Show
          Arun C Murthy added a comment - Alejandro, I think Giri's comment makes sense - for now having hadoop-yarn in hadoop-mapreduce helps folks looking for a single checkout... so, we could keep it there. Thoughts?
          Hide
          Alejandro Abdelnur added a comment -

          Having v1 and v2 patches people can see how thing would look in a flat and hierarchical worlds. Please try both.

          Show
          Alejandro Abdelnur added a comment - Having v1 and v2 patches people can see how thing would look in a flat and hierarchical worlds. Please try both.
          Hide
          Arun C Murthy added a comment -

          Awesome, v2 works flawlessly.

          I'm +1 on v2.

          Show
          Arun C Murthy added a comment - Awesome, v2 works flawlessly. I'm +1 on v2.
          Hide
          Mahadev konar added a comment -

          v2 looks good to me! +1 for the v2 patch.

          Show
          Mahadev konar added a comment - v2 looks good to me! +1 for the v2 patch.
          Hide
          Eric Yang added a comment -

          +1 on v2. Thanks

          Show
          Eric Yang added a comment - +1 on v2. Thanks
          Hide
          Tom White added a comment -

          I prefer v1, however I think reasonable people can differ on flat vs. nested, and there is no overwhelming consensus on other Maven projects for one as far as I know. Having consistency is more important, and either one of these patches will improve that.

          Show
          Tom White added a comment - I prefer v1, however I think reasonable people can differ on flat vs. nested, and there is no overwhelming consensus on other Maven projects for one as far as I know. Having consistency is more important, and either one of these patches will improve that.
          Hide
          Arun C Murthy added a comment -

          I'm going to commit this unless anyone has reservations...

          Show
          Arun C Murthy added a comment - I'm going to commit this unless anyone has reservations...
          Hide
          Arun C Murthy added a comment -

          I just committed this. Thanks Alejandro!

          Show
          Arun C Murthy added a comment - I just committed this. Thanks Alejandro!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #782 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/782/)
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
          • /hadoop/common/trunk/hadoop-annotations
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
          • /hadoop/common/trunk/hadoop-alfredo
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
          • /hadoop/common/trunk/hadoop-mapreduce-project
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-common-project
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth
          • /hadoop/common/trunk/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-mapreduce
          • /hadoop/common/trunk/hadoop-common
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #782 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/782/ ) HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335 Files : /hadoop/common/trunk/hadoop-hdfs-project/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-common-project/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332 Files : /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client /hadoop/common/trunk/hadoop-annotations /hadoop/common/trunk/hadoop-common-project/hadoop-annotations /hadoop/common/trunk/hadoop-alfredo /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client /hadoop/common/trunk/hadoop-mapreduce-project /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/trunk/hadoop-common-project /hadoop/common/trunk/hadoop-common-project/hadoop-auth /hadoop/common/trunk/hadoop-hdfs /hadoop/common/trunk/hadoop-mapreduce /hadoop/common/trunk/hadoop-common /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #779 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/779/)
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
          • /hadoop/common/trunk/hadoop-annotations
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
          • /hadoop/common/trunk/hadoop-alfredo
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
          • /hadoop/common/trunk/hadoop-mapreduce-project
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-common-project
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth
          • /hadoop/common/trunk/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-mapreduce
          • /hadoop/common/trunk/hadoop-common
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #779 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/779/ ) HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335 Files : /hadoop/common/trunk/hadoop-hdfs-project/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-common-project/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332 Files : /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client /hadoop/common/trunk/hadoop-annotations /hadoop/common/trunk/hadoop-common-project/hadoop-annotations /hadoop/common/trunk/hadoop-alfredo /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client /hadoop/common/trunk/hadoop-mapreduce-project /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/trunk/hadoop-common-project /hadoop/common/trunk/hadoop-common-project/hadoop-auth /hadoop/common/trunk/hadoop-hdfs /hadoop/common/trunk/hadoop-mapreduce /hadoop/common/trunk/hadoop-common /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #857 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/857/)
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
          • /hadoop/common/trunk/hadoop-annotations
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
          • /hadoop/common/trunk/hadoop-alfredo
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
          • /hadoop/common/trunk/hadoop-mapreduce-project
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-common-project
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth
          • /hadoop/common/trunk/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-mapreduce
          • /hadoop/common/trunk/hadoop-common
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #857 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/857/ ) HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335 Files : /hadoop/common/trunk/hadoop-hdfs-project/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-common-project/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332 Files : /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client /hadoop/common/trunk/hadoop-annotations /hadoop/common/trunk/hadoop-common-project/hadoop-annotations /hadoop/common/trunk/hadoop-alfredo /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client /hadoop/common/trunk/hadoop-mapreduce-project /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/trunk/hadoop-common-project /hadoop/common/trunk/hadoop-common-project/hadoop-auth /hadoop/common/trunk/hadoop-hdfs /hadoop/common/trunk/hadoop-mapreduce /hadoop/common/trunk/hadoop-common /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #787 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/787/)
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
          • /hadoop/common/trunk/hadoop-annotations
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
          • /hadoop/common/trunk/hadoop-alfredo
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
          • /hadoop/common/trunk/hadoop-mapreduce-project
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-common-project
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth
          • /hadoop/common/trunk/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-mapreduce
          • /hadoop/common/trunk/hadoop-common
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #787 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/787/ ) HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335 Files : /hadoop/common/trunk/hadoop-hdfs-project/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-common-project/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332 Files : /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client /hadoop/common/trunk/hadoop-annotations /hadoop/common/trunk/hadoop-common-project/hadoop-annotations /hadoop/common/trunk/hadoop-alfredo /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client /hadoop/common/trunk/hadoop-mapreduce-project /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/trunk/hadoop-common-project /hadoop/common/trunk/hadoop-common-project/hadoop-auth /hadoop/common/trunk/hadoop-hdfs /hadoop/common/trunk/hadoop-mapreduce /hadoop/common/trunk/hadoop-common /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #765 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/765/)
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
          HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml

          acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
          Files :

          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
          • /hadoop/common/trunk/hadoop-annotations
          • /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
          • /hadoop/common/trunk/hadoop-alfredo
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
          • /hadoop/common/trunk/hadoop-mapreduce-project
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-common-project
          • /hadoop/common/trunk/hadoop-common-project/hadoop-auth
          • /hadoop/common/trunk/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-mapreduce
          • /hadoop/common/trunk/hadoop-common
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #765 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/765/ ) HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. HADOOP-7560 . Change src layout to be heirarchical. Contributed by Alejandro Abdelnur. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335 Files : /hadoop/common/trunk/hadoop-hdfs-project/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml /hadoop/common/trunk/hadoop-common-project/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332 Files : /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client /hadoop/common/trunk/hadoop-annotations /hadoop/common/trunk/hadoop-common-project/hadoop-annotations /hadoop/common/trunk/hadoop-alfredo /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client /hadoop/common/trunk/hadoop-mapreduce-project /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs /hadoop/common/trunk/hadoop-common-project /hadoop/common/trunk/hadoop-common-project/hadoop-auth /hadoop/common/trunk/hadoop-hdfs /hadoop/common/trunk/hadoop-mapreduce /hadoop/common/trunk/hadoop-common /hadoop/common/trunk/hadoop-common-project/hadoop-common
          Hide
          Thomas Graves added a comment -

          It doesn't looks like BUILDING.txt was updated with this, can you please update.

          Show
          Thomas Graves added a comment - It doesn't looks like BUILDING.txt was updated with this, can you please update.

            People

            • Assignee:
              Alejandro Abdelnur
              Reporter:
              Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development