Hadoop Common
  1. Hadoop Common
  2. HADOOP-6436

Remove auto-generated native build files

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0, 0.23.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      The native build run when from trunk now requires autotools, libtool and openssl dev libraries.

      Description

      The repo currently includes the automake and autoconf generated files for the native build. Per discussion on HADOOP-6421 let's remove them and use the host's automake and autoconf. We should also do this for libhdfs and fuse-dfs.

      1. 6436.patch
        1.33 MB
        Roman Shaposhnik

        Issue Links

          Activity

          Hide
          Philip Zeyliger added a comment -

          +1.

          Show
          Philip Zeyliger added a comment - +1.
          Hide
          Todd Lipcon added a comment -

          +1, but I think building a "dist" style package should involve generating the files (like make dist does for normal non-Java projects)

          Show
          Todd Lipcon added a comment - +1, but I think building a "dist" style package should involve generating the files (like make dist does for normal non-Java projects)
          Hide
          Allen Wittenauer added a comment -

          +1 to what Todd said. Not all operating systems ship the GNU toolchain.

          Show
          Allen Wittenauer added a comment - +1 to what Todd said. Not all operating systems ship the GNU toolchain.
          Hide
          Matt Massie added a comment -

          +1

          Agree with Todd and Allen that the final artifact should be a "dist" (or "distcheck") generated tarball with no dependencies on the GNU toolchain.

          Show
          Matt Massie added a comment - +1 Agree with Todd and Allen that the final artifact should be a "dist" (or "distcheck") generated tarball with no dependencies on the GNU toolchain.
          Hide
          Roman Shaposhnik added a comment -

          I'm attaching a first cut at the patch. Please let me know what do you think about the approach taken.

          Notes:

          1. package target is modified because the logic here is that we want those files to end up in the tarball before a release. As was noted by Todd on the original issue we want those files to be bundled for platforms that lack autotools.

          2. the build (at least the release build) now depends on the entire autoconf

          Show
          Roman Shaposhnik added a comment - I'm attaching a first cut at the patch. Please let me know what do you think about the approach taken. Notes: 1. package target is modified because the logic here is that we want those files to end up in the tarball before a release. As was noted by Todd on the original issue we want those files to be bundled for platforms that lack autotools. 2. the build (at least the release build) now depends on the entire autoconf
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12468618/6436.patch
          against trunk revision 1060632.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these core unit tests:

          -1 contrib tests. The patch failed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//testReport/
          Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12468618/6436.patch against trunk revision 1060632. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//testReport/ Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/186//console This message is automatically generated.
          Hide
          Eli Collins added a comment -

          The change looks good. The diff to src/native/config.h needs to be updated, thinks it's a reverse patch. What testing has been done, eg run the codec test with native libs enabled?

          Show
          Eli Collins added a comment - The change looks good. The diff to src/native/config.h needs to be updated, thinks it's a reverse patch. What testing has been done, eg run the codec test with native libs enabled?
          Hide
          Roman Shaposhnik added a comment -

          This has been tested via testing a corresponding MAPREDUCE change (MAPREDUCE-2260) on a 64bit Linux box

          Show
          Roman Shaposhnik added a comment - This has been tested via testing a corresponding MAPREDUCE change ( MAPREDUCE-2260 ) on a 64bit Linux box
          Hide
          Eli Collins added a comment -

          +1

          I verified ant -Dcompile.c++=true compile-native builds correctly on 64-bit Centos 5 and 64-bit Ubuntu Maverick. And that ant create-native-configure -Dneed.native.configure=true can be used to regenerate the configure file.

          I will commit this to trunk and branch 22. Thanks Roman!

          Show
          Eli Collins added a comment - +1 I verified ant -Dcompile.c++=true compile-native builds correctly on 64-bit Centos 5 and 64-bit Ubuntu Maverick. And that ant create-native-configure -Dneed.native.configure=true can be used to regenerate the configure file. I will commit this to trunk and branch 22. Thanks Roman!
          Hide
          Eli Collins added a comment -

          I forgot to mention, you may need to restart your host after installing libtool if libtool is not present.

          Show
          Eli Collins added a comment - I forgot to mention, you may need to restart your host after installing libtool if libtool is not present.
          Hide
          Eli Collins added a comment -

          I've committed this.

          Show
          Eli Collins added a comment - I've committed this.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-22-branch #22 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/22/)
          HADOOP-6436. svn merge -c 1065858 from trunk

          Show
          Hudson added a comment - Integrated in Hadoop-Common-22-branch #22 (See https://hudson.apache.org/hudson/job/Hadoop-Common-22-branch/22/ ) HADOOP-6436 . svn merge -c 1065858 from trunk
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #492 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/492/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #492 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/492/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #593 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/593/)
          HADOOP-6436. Remove auto-generated native build files. Contributed by Roman Shaposhnik

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #593 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/593/ ) HADOOP-6436 . Remove auto-generated native build files. Contributed by Roman Shaposhnik

            People

            • Assignee:
              Roman Shaposhnik
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development