Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8368

Use CMake rather than autotools to build native code

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      It would be good to use cmake rather than autotools to build the native (C/C++) code in Hadoop.

      Rationale:
      1. automake depends on shell scripts, which often have problems running on different operating systems. It would be extremely difficult, and perhaps impossible, to use autotools under Windows. Even if it were possible, it might require horrible workarounds like installing cygwin. Even on Linux variants like Ubuntu 12.04, there are major build issues because /bin/sh is the Dash shell, rather than the Bash shell as it is in other Linux versions. It is currently impossible to build the native code under Ubuntu 12.04 because of this problem.

      CMake has robust cross-platform support, including Windows. It does not use shell scripts.

      2. automake error messages are very confusing. For example, "autoreconf: cannot empty /tmp/ar0.4849: Is a directory" or "Can't locate object method "path" via package "Autom4te..." are common error messages. In order to even start debugging automake problems you need to learn shell, m4, sed, and the a bunch of other things. With CMake, all you have to learn is the syntax of CMakeLists.txt, which is simple.

      CMake can do all the stuff autotools can, such as making sure that required libraries are installed. There is a Maven plugin for CMake as well.

      3. Different versions of autotools can have very different behaviors. For example, the version installed under openSUSE defaults to putting libraries in /usr/local/lib64, whereas the version shipped with Ubuntu 11.04 defaults to installing the same libraries under /usr/local/lib. (This is why the FUSE build is currently broken when using OpenSUSE.) This is another source of build failures and complexity. If things go wrong, you will often get an error message which is incomprehensible to normal humans (see point #2).

      CMake allows you to specify the minimum_required_version of CMake that a particular CMakeLists.txt will accept. In addition, CMake maintains strict backwards compatibility between different versions. This prevents build bugs due to version skew.

      4. autoconf, automake, and libtool are large and rather slow. This adds to build time.

      For all these reasons, I think we should switch to CMake for compiling native (C/C++) code in Hadoop.

        Attachments

        1. HADOOP-8368.001.patch
          34 kB
          Colin P. McCabe
        2. HADOOP-8368.005.patch
          444 kB
          Colin P. McCabe
        3. HADOOP-8368.006.patch
          445 kB
          Colin P. McCabe
        4. HADOOP-8368.007.patch
          445 kB
          Colin P. McCabe
        5. HADOOP-8368.008.patch
          445 kB
          Colin P. McCabe
        6. HADOOP-8368.009.patch
          443 kB
          Colin P. McCabe
        7. HADOOP-8368.010.patch
          443 kB
          Colin P. McCabe
        8. HADOOP-8368.012.half.patch
          221 kB
          Colin P. McCabe
        9. HADOOP-8368.012.patch
          445 kB
          Colin P. McCabe
        10. HADOOP-8368.012.rm.patch
          3 kB
          Colin P. McCabe
        11. HADOOP-8368.014.trimmed.patch
          53 kB
          Colin P. McCabe
        12. HADOOP-8368.015.trimmed.patch
          53 kB
          Colin P. McCabe
        13. HADOOP-8368.016.trimmed.patch
          65 kB
          Colin P. McCabe
        14. HADOOP-8368.018.trimmed.patch
          65 kB
          Colin P. McCabe
        15. HADOOP-8368.020.rm.patch
          2 kB
          Colin P. McCabe
        16. HADOOP-8368.020.trimmed.patch
          50 kB
          Colin P. McCabe
        17. HADOOP-8368.021.trimmed.patch
          50 kB
          Colin P. McCabe
        18. HADOOP-8368.023.trimmed.patch
          58 kB
          Colin P. McCabe
        19. HADOOP-8368.024.trimmed.patch
          56 kB
          Colin P. McCabe
        20. HADOOP-8368.025.trimmed.patch
          56 kB
          Colin P. McCabe
        21. HADOOP-8368.026.rm.patch
          2 kB
          Colin P. McCabe
        22. HADOOP-8368.026.trimmed.patch
          47 kB
          Colin P. McCabe
        23. HADOOP-8368.028.rm.patch
          2 kB
          Colin P. McCabe
        24. HADOOP-8368.028.trimmed.patch
          44 kB
          Colin P. McCabe
        25. HADOOP-8368.029.patch
          102 kB
          Colin P. McCabe
        26. HADOOP-8368.030.patch
          101 kB
          Colin P. McCabe
        27. HADOOP-8368.030.patch
          101 kB
          Colin P. McCabe
        28. HADOOP-8368.030.rm.patch
          2 kB
          Colin P. McCabe
        29. HADOOP-8368.030.trimmed.patch
          44 kB
          Colin P. McCabe
        30. HADOOP-8368-b2.001.patch
          259 kB
          Colin P. McCabe
        31. HADOOP-8368-b2.001.rm.patch
          2 kB
          Colin P. McCabe
        32. HADOOP-8368-b2.001.trimmed.patch
          47 kB
          Colin P. McCabe
        33. HADOOP-8368-b2.002.rm.patch
          2 kB
          Colin P. McCabe
        34. HADOOP-8368-b2.002.trimmed.patch
          43 kB
          Colin P. McCabe
        35. HADOOP-8368-b2.003.rm.patch
          2 kB
          Colin P. McCabe
        36. HADOOP-8368-b2.003.trimmed.patch
          43 kB
          Colin P. McCabe

          Issue Links

            Activity

              People

              • Assignee:
                cmccabe Colin P. McCabe
                Reporter:
                cmccabe Colin P. McCabe
              • Votes:
                0 Vote for this issue
                Watchers:
                22 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: