Hadoop Common
  1. Hadoop Common
  2. HADOOP-6436

Remove auto-generated native build files

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0, 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      The native build run when from trunk now requires autotools, libtool and openssl dev libraries.

      Description

      The repo currently includes the automake and autoconf generated files for the native build. Per discussion on HADOOP-6421 let's remove them and use the host's automake and autoconf. We should also do this for libhdfs and fuse-dfs.

      1. 6436.patch
        1.33 MB
        Roman Shaposhnik

        Issue Links

          Activity

          Eli Collins created issue -
          Eli Collins made changes -
          Field Original Value New Value
          Assignee Eli Collins [ eli ]
          Hide
          Philip Zeyliger added a comment -

          +1.

          Show
          Philip Zeyliger added a comment - +1.
          Hide
          Todd Lipcon added a comment -

          +1, but I think building a "dist" style package should involve generating the files (like make dist does for normal non-Java projects)

          Show
          Todd Lipcon added a comment - +1, but I think building a "dist" style package should involve generating the files (like make dist does for normal non-Java projects)
          Todd Lipcon made changes -
          Link This issue relates to HADOOP-6069 [ HADOOP-6069 ]
          Eli Collins made changes -
          Assignee Eli Collins [ eli ]
          Hide
          Allen Wittenauer added a comment -

          +1 to what Todd said. Not all operating systems ship the GNU toolchain.

          Show
          Allen Wittenauer added a comment - +1 to what Todd said. Not all operating systems ship the GNU toolchain.
          Roman Shaposhnik made changes -
          Link This issue incorporates HDFS-1582 [ HDFS-1582 ]
          Roman Shaposhnik made changes -
          Link This issue incorporates MAPREDUCE-2260 [ MAPREDUCE-2260 ]
          Hide
          Matt Massie added a comment -

          +1

          Agree with Todd and Allen that the final artifact should be a "dist" (or "distcheck") generated tarball with no dependencies on the GNU toolchain.

          Show
          Matt Massie added a comment - +1 Agree with Todd and Allen that the final artifact should be a "dist" (or "distcheck") generated tarball with no dependencies on the GNU toolchain.
          Hide
          Roman Shaposhnik added a comment -

          I'm attaching a first cut at the patch. Please let me know what do you think about the approach taken.

          Notes:

          1. package target is modified because the logic here is that we want those files to end up in the tarball before a release. As was noted by Todd on the original issue we want those files to be bundled for platforms that lack autotools.

          2. the build (at least the release build) now depends on the entire autoconf

          Show
          Roman Shaposhnik added a comment - I'm attaching a first cut at the patch. Please let me know what do you think about the approach taken. Notes: 1. package target is modified because the logic here is that we want those files to end up in the tarball before a release. As was noted by Todd on the original issue we want those files to be bundled for platforms that lack autotools. 2. the build (at least the release build) now depends on the entire autoconf
          Roman Shaposhnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Roman Shaposhnik made changes -
          Attachment 6436.patch [ 12468618 ]
          Roman Shaposhnik made changes -
          Assignee Roman Shaposhnik [ rvs ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12468618/6436.patch
          against trunk revision 1060632.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:

          -1 contrib tests. The patch failed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12468618/6436.patch against trunk revision 1060632. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          The change looks good. The diff to src/native/config.h needs to be updated, thinks it's a reverse patch. What testing has been done, eg run the codec test with native libs enabled?

          Show
          Eli Collins added a comment - The change looks good. The diff to src/native/config.h needs to be updated, thinks it's a reverse patch. What testing has been done, eg run the codec test with native libs enabled?
          Hide
          Roman Shaposhnik added a comment -

          This has been tested via testing a corresponding MAPREDUCE change (MAPREDUCE-2260) on a 64bit Linux box

          Show
          Roman Shaposhnik added a comment - This has been tested via testing a corresponding MAPREDUCE change ( MAPREDUCE-2260 ) on a 64bit Linux box
          Hide
          Eli Collins added a comment -

          +1

          I verified ant -Dcompile.c++=true compile-native builds correctly on 64-bit Centos 5 and 64-bit Ubuntu Maverick. And that ant create-native-configure -Dneed.native.configure=true can be used to regenerate the configure file.

          I will commit this to trunk and branch 22. Thanks Roman!

          Show
          Eli Collins added a comment - +1 I verified ant -Dcompile.c++=true compile-native builds correctly on 64-bit Centos 5 and 64-bit Ubuntu Maverick. And that ant create-native-configure -Dneed.native.configure=true can be used to regenerate the configure file. I will commit this to trunk and branch 22. Thanks Roman!
          Eli Collins made changes -
          Hadoop Flags [Reviewed]
          Release Note The native build run when from trunk now requires autotools, libtool and openssl dev libraries.
          Fix Version/s 0.22.0 [ 12314296 ]
          Fix Version/s 0.23.0 [ 12315569 ]
          Hide
          Eli Collins added a comment -

          I forgot to mention, you may need to restart your host after installing libtool if libtool is not present.

          Show
          Eli Collins added a comment - I forgot to mention, you may need to restart your host after installing libtool if libtool is not present.
          Hide
          Eli Collins added a comment -

          I've committed this.

          Show
          Eli Collins added a comment - I've committed this.
          Eli Collins made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Eli Collins made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-22-branch #22 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/22/)
          HADOOP-6436. svn merge -c 1065858 from trunk

          Show
          Hudson added a comment - Integrated in Hadoop-Common-22-branch #22 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/22/ ) HADOOP-6436 . svn merge -c 1065858 from trunk
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #492 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/492/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #492 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/492/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #593 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/593/)
          HADOOP-6436. Remove auto-generated native build files. Contributed by Roman Shaposhnik

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #593 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/593/ ) HADOOP-6436 . Remove auto-generated native build files. Contributed by Roman Shaposhnik
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          402d 8h 29m 1 Roman Shaposhnik 18/Jan/11 02:30
          Patch Available Patch Available Resolved Resolved
          14d 2h 49m 1 Eli Collins 01/Feb/11 05:19
          Resolved Resolved Closed Closed
          9s 1 Eli Collins 01/Feb/11 05:19

            People

            • Assignee:
              Roman Shaposhnik
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development