Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14064

Native maven build fails for non-default zlib location

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Flags:
      Patch

      Description

      This bug prevents me from building a native Hadoop for my system.

      The current build fails for me if zlib is not located the usual location(s).

      I looked into ways to pass the necessary information to the Maven build. I found nothing that worked short of modifying the pom.xml of hadoop-common.

      Luckily, this is easy for the project maintainers to fix – see patch below – since the build already includes the FindZLIB module for cmake (https://cmake.org/cmake/help/v3.0/module/FindZLIB.html). All that is missing is to allow maven command line parameters pass through to FindZLIB.

      I suggest the following patch (works for me) that accepts two command line arguments, `zlib.lib` and `zlib.include` and passes them to cmake as ZLIB_LIBRARY and ZLIB_INCLUDE_DIR, respectively.

      It would be used like this:

      mvn package -Pdist,native -DskipTests -Dtar \
      -Dzlib.lib=$HOME/lib/libz.so \
      -Dzlib.include=$HOME/include
      

      Here is my suggested patch:

      diff -r hadoop-2.7.3-src/hadoop-common-project/hadoop-common/pom.xml hadoop-2.7.3-src-patched/hadoop-common-project/hadoop-common/pom.xml
      521a522,523
      >         <zlib.lib></zlib.lib>
      >         <zlib.include></zlib.include>
      605c607
      <                       <arg line="${basedir}/src/ -DGENERATED_JAVAH=${project.build.directory}/native/javah -DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DREQUIRE_BZIP2=${require.bzip2} -DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix} -DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include} -DREQUIRE_OPENSSL=${require.openssl} -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix} -DCUSTOM_OPENSSL_LIB=${openssl.lib} -DCUSTOM_OPENSSL_INCLUDE=${openssl.include} -DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
      ---
      >                       <arg line="${basedir}/src/ -DGENERATED_JAVAH=${project.build.directory}/native/javah -DJVM_ARCH_DATA_MODEL=${sun.arch.data.model} -DZLIB_LIBRARY=${zlib.lib} -DZLIB_INCLUDE_DIR=${zlib.include} -DREQUIRE_BZIP2=${require.bzip2} -DREQUIRE_SNAPPY=${require.snappy} -DCUSTOM_SNAPPY_PREFIX=${snappy.prefix} -DCUSTOM_SNAPPY_LIB=${snappy.lib} -DCUSTOM_SNAPPY_INCLUDE=${snappy.include} -DREQUIRE_OPENSSL=${require.openssl} -DCUSTOM_OPENSSL_PREFIX=${openssl.prefix} -DCUSTOM_OPENSSL_LIB=${openssl.lib} -DCUSTOM_OPENSSL_INCLUDE=${openssl.include} -DEXTRA_LIBHADOOP_RPATH=${extra.libhadoop.rpath}"/>
      643a646,647
      >         <zlib.lib></zlib.lib>
      >         <zlib.include></zlib.include>
      

      Suggestions and improvements are welcome.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              david.james David James
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: