Hadoop Common
  1. Hadoop Common
  2. HADOOP-6305

Unify build property names to facilitate cross-projects modifications

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Current build files of common, mapreduce, hdfs have their own unique ways of naming properties which otherwise have very same meaning. I.e. the locations of java source code are called core.src.dir, mapred.src.dir, and hdfs.src.dir respectively.

      It makes modifications needed for all three projects to be very unique thus increasing the number of difference between builds of the projects.

      1. HADOOP-6305.patch
        7 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          Konstantin Boudnik added a comment -

          This patch addresses the issue for Common subproject. test-patch result to follow shortly.

          Show
          Konstantin Boudnik added a comment - This patch addresses the issue for Common subproject. test-patch result to follow shortly.
          Hide
          Konstantin Boudnik added a comment -

          Without this fix back porting of fault injection framework followed by reusing it from Common will be quite painful

          Show
          Konstantin Boudnik added a comment - Without this fix back porting of fault injection framework followed by reusing it from Common will be quite painful
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Should we also remove core from test.core.build.classes?

          @@ -171,7 +171,7 @@
               <path refid="ivy-common.classpath"/>
             </path>
           
          -  <path id="test.core.classpath">
          +  <path id="test.classpath">
               <pathelement location="${test.build.extraconf}"/>
               <pathelement location="${test.core.build.classes}" />
               <pathelement location="${test.src.dir}"/>
          

          Filed HADOOP-6159 some time ago to remove "core" from build.xml.

          Show
          Tsz Wo Nicholas Sze added a comment - Should we also remove core from test.core.build.classes? @@ -171,7 +171,7 @@ <path refid="ivy-common.classpath"/> </path> - <path id="test.core.classpath"> + <path id="test.classpath"> <pathelement location="${test.build.extraconf}"/> <pathelement location="${test.core.build.classes}" /> <pathelement location="${test.src.dir}"/> Filed HADOOP-6159 some time ago to remove "core" from build.xml.
          Hide
          Konstantin Boudnik added a comment -

          It seems to be a separate problem: there's test.core.build.classes and test.build.classes
          The former is used to as the source of class files for the latter one. I.e.

            <target name="jar-test" depends="compile-core-test" description="Make hadoop-test.jar">
              <copy todir="${test.build.classes}">
                <fileset dir="${test.core.build.classes}"/>
              </copy>
          

          While it certainly seems inappropriate it ought to be addressed by a separate JIRA such your HADOOP-6159. Does it make sense?

          Show
          Konstantin Boudnik added a comment - It seems to be a separate problem: there's test.core.build.classes and test.build.classes The former is used to as the source of class files for the latter one. I.e. <target name="jar-test" depends="compile-core-test" description="Make hadoop-test.jar"> <copy todir="${test.build.classes}"> <fileset dir="${test.core.build.classes}"/> </copy> While it certainly seems inappropriate it ought to be addressed by a separate JIRA such your HADOOP-6159 . Does it make sense?
          Hide
          Konstantin Boudnik added a comment -

          I've ran test-patch locally

          There appear to be 0 release audit warnings before the patch and 0 release audit warnings after applying the patch.
          +1 overall.  
              +1 @author.  The patch does not contain any @author tags.
              -1 tests included.  The patch doesn't appear to include any new or modified tests.
                                  Please justify why no new tests are needed for this patch.
                                  Also please list what manual steps were performed to verify this patch.
              +1 javadoc.  The javadoc tool did not generate any warning messages.
              +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
              +1 findbugs.  The patch does not introduce any new Findbugs warnings.
              +1 release audit.  The applied patch does not increase the total number of release audit warnings.
          ======================================================================
          ======================================================================
              Finished build.
          ======================================================================
          ======================================================================
          

          The patch doesn't require any tests for it is an build's internal change.

          Show
          Konstantin Boudnik added a comment - I've ran test-patch locally There appear to be 0 release audit warnings before the patch and 0 release audit warnings after applying the patch. +1 overall. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. ====================================================================== ====================================================================== Finished build. ====================================================================== ====================================================================== The patch doesn't require any tests for it is an build's internal change.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > While it certainly seems inappropriate it ought to be addressed by a separate JIRA such your HADOOP-6159. Does it make sense?

          Sure.

          +1 patch looks good. Could you also try various ant targets, especially "ant tar", to make sure everything still working?

          PS: Not sure if this is qualified to be committed to 0.21.

          Show
          Tsz Wo Nicholas Sze added a comment - > While it certainly seems inappropriate it ought to be addressed by a separate JIRA such your HADOOP-6159 . Does it make sense? Sure. +1 patch looks good. Could you also try various ant targets, especially "ant tar", to make sure everything still working? PS: Not sure if this is qualified to be committed to 0.21.
          Hide
          Konstantin Boudnik added a comment -
          • log of ant tar:
            Buildfile: build.xml
            
            clover.setup:
            ...
            compile:
            
            package:
                [mkdir] Created dir: /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/contrib/hod
                 [copy] Copying 70 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/contrib/hod
                 [copy] Copying 2 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev
                 [copy] Copying 18 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/bin
                 [copy] Copying 15 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/conf
                 [copy] Copying 1212 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/docs
                 [copy] Copied 99 empty directories to 6 empty directories under /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/docs
                 [copy] Copying 1 file to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev
                 [copy] Copying 4 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/ivy
                 [copy] Copying 4 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev
                 [copy] Copying 989 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/src
                 [copy] Copied 174 empty directories to 5 empty directories under /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/src
                 [copy] Copying 1 file to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev
            
            tar:
                  [tar] Building tar: /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev.tar.gz
            
            BUILD SUCCESSFUL
            Total time: 2 minutes 46 seconds
            
            

          Waiting on Nigel's comment about committing it to 0.21 and will commit it accordingly after.

          Show
          Konstantin Boudnik added a comment - log of ant tar : Buildfile: build.xml clover.setup: ... compile: package: [mkdir] Created dir: /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/contrib/hod [copy] Copying 70 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/contrib/hod [copy] Copying 2 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev [copy] Copying 18 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/bin [copy] Copying 15 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/conf [copy] Copying 1212 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/docs [copy] Copied 99 empty directories to 6 empty directories under /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/docs [copy] Copying 1 file to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev [copy] Copying 4 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/ivy [copy] Copying 4 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev [copy] Copying 989 files to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/src [copy] Copied 174 empty directories to 5 empty directories under /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev/src [copy] Copying 1 file to /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev tar: [tar] Building tar: /Users/xxx/work/Common.test/build/hadoop-core-0.22.0-dev.tar.gz BUILD SUCCESSFUL Total time: 2 minutes 46 seconds Waiting on Nigel's comment about committing it to 0.21 and will commit it accordingly after.
          Hide
          Nigel Daley added a comment -

          PS: Not sure if this is qualified to be committed to 0.21.

          There is no restriction on backporting build/test/doc related patches.

          Show
          Nigel Daley added a comment - PS: Not sure if this is qualified to be committed to 0.21. There is no restriction on backporting build/test/doc related patches.
          Hide
          Konstantin Boudnik added a comment -

          I've just committed this.

          Show
          Konstantin Boudnik added a comment - I've just committed this.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #58 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/58/)
          . Unify build property names to facilitate cross-projects modifications. Contributed by Konstantin Boudnik

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #58 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/58/ ) . Unify build property names to facilitate cross-projects modifications. Contributed by Konstantin Boudnik
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #123 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/123/)
          . Unify build property names to facilitate cross-projects modifications. Contributed by Konstantin Boudnik

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #123 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/123/ ) . Unify build property names to facilitate cross-projects modifications. Contributed by Konstantin Boudnik

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development