Hadoop Common
  1. Hadoop Common
  2. HADOOP-9162

Add utility to check native library availability

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.3-alpha, 3.0.0
    • Fix Version/s: 2.0.3-alpha
    • Component/s: native
    • Labels:
      None

      Description

      Many times, after deploy hadoop or when trouble shooting, we need to check whether native library(along with native compression libraries) can work properly, and I just want to use one command to check that, like this:

      hadoop org.apache.hadoop.util.NativeCodeLoader

      and it shows:

      Native library loading test:
      hadoop: false
      zlib: false
      snappy: false
      lz4: false

      1. HADOOP-9162.v4.patch
        7 kB
        Binglin Chang
      2. HADOOP-9162.v3.patch
        6 kB
        Binglin Chang
      3. HADOOP-9162.v2.patch
        4 kB
        Binglin Chang
      4. HADOOP-9162.patch
        2 kB
        Binglin Chang

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1293 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1293/)
          HADOOP-9162. Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1293 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1293/ ) HADOOP-9162 . Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1263 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1263/)
          HADOOP-9162. Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390)

          Result = FAILURE
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1263 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1263/ ) HADOOP-9162 . Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #74 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/74/)
          HADOOP-9162. Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Show
          Hudson added a comment - Integrated in Hadoop-Yarn-trunk #74 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/74/ ) HADOOP-9162 . Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Hide
          Suresh Srinivas added a comment -

          I committed the patch trunk and branch-2. Thank you Binglin.

          Show
          Suresh Srinivas added a comment - I committed the patch trunk and branch-2. Thank you Binglin.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #3157 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3157/)
          HADOOP-9162. Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390)

          Result = SUCCESS
          suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Show
          Hudson added a comment - Integrated in Hadoop-trunk-Commit #3157 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3157/ ) HADOOP-9162 . Add utility to check native library availability. Contributed by Binglin Chang. (Revision 1425390) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1425390 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/NativeLibraryChecker.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeLibraryChecker.java
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12562192/HADOOP-9162.v4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1923//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1923//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562192/HADOOP-9162.v4.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1923//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1923//console This message is automatically generated.
          Hide
          Suresh Srinivas added a comment -

          We add those annotations to all the public classes. Nothing prevents people from adding methods to that class in the future and then it gets used by user apps. So it is better to have the annotation for all the public classes.

          Show
          Suresh Srinivas added a comment - We add those annotations to all the public classes. Nothing prevents people from adding methods to that class in the future and then it gets used by user apps. So it is better to have the annotation for all the public classes.
          Hide
          Luke Lu added a comment -

          What would be the semantics of InterfaceAudience.Private on a executable main class? The class name can only be referred in a Hadoop script and not some other scripts?

          Show
          Luke Lu added a comment - What would be the semantics of InterfaceAudience.Private on a executable main class? The class name can only be referred in a Hadoop script and not some other scripts?
          Hide
          Suresh Srinivas added a comment -

          Minor Comments:

          1. Please add annotation to NativeLibraryChecker - InterfaceAudience.Private
          2. Please add description of -a to the usage.
          3. Do you think it is a good idea to have checknative [hadoop|zlib|snappy|lz4] and absence of specific option prints for all? I am okay without this change, if you choose not to make this change.
          Show
          Suresh Srinivas added a comment - Minor Comments: Please add annotation to NativeLibraryChecker - InterfaceAudience.Private Please add description of -a to the usage. Do you think it is a good idea to have checknative [hadoop|zlib|snappy|lz4] and absence of specific option prints for all? I am okay without this change, if you choose not to make this change.
          Hide
          Luke Lu added a comment -

          The v3 patch lgtm. +1.

          Show
          Luke Lu added a comment - The v3 patch lgtm. +1.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12562104/HADOOP-9162.v3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1920//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1920//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562104/HADOOP-9162.v3.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1920//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1920//console This message is automatically generated.
          Hide
          Binglin Chang added a comment -

          I don't see an easy cross platform way either -unless the libraries themselves return a version number. We could add that to the Hadoop-specific codecs, but not to Snappy.

          Add library version number and/or library file to native libraries are not in the scope of this JIRA, create HADOOP-9164 for this.

          Show
          Binglin Chang added a comment - I don't see an easy cross platform way either -unless the libraries themselves return a version number. We could add that to the Hadoop-specific codecs, but not to Snappy. Add library version number and/or library file to native libraries are not in the scope of this JIRA, create HADOOP-9164 for this.
          Hide
          Binglin Chang added a comment -

          Using ExitUtil and add unittest.

          Show
          Binglin Chang added a comment - Using ExitUtil and add unittest.
          Hide
          Steve Loughran added a comment -

          I don't know how to achieve that, it seams a problem to introspect System.loadLibrary("hadoop"); to find out which file it actually load.

          I don't see an easy cross platform way either -unless the libraries themselves return a version number. We could add that to the Hadoop-specific codecs, but not to Snappy.

          Regarding tests, if you use ExitUtils.terminate() instead of System.exit(), you can write a test which calls the main() function with different args, -after disabling exit with ExitUtils.disableSystemExit() -this will cause an ExitException to be thrown instead of the process exiting. That would let you write some unit tests that check the class behaviour.

          Show
          Steve Loughran added a comment - I don't know how to achieve that, it seams a problem to introspect System.loadLibrary("hadoop"); to find out which file it actually load. I don't see an easy cross platform way either -unless the libraries themselves return a version number. We could add that to the Hadoop-specific codecs, but not to Snappy. Regarding tests, if you use ExitUtils.terminate() instead of System.exit() , you can write a test which calls the main() function with different args, -after disabling exit with ExitUtils.disableSystemExit() -this will cause an ExitException to be thrown instead of the process exiting. That would let you write some unit tests that check the class behaviour.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12562050/HADOOP-9162.v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1917//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1917//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12562050/HADOOP-9162.v2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1917//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1917//console This message is automatically generated.
          Hide
          Binglin Chang added a comment -

          new version addressing my previous comments.
          As a single main class util, it's hard to add unit test. But I have done some simple manual test, here is the result.

          decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop 
          Usage: hadoop [--config confdir] COMMAND
                 where COMMAND is one of:
            fs                   run a generic filesystem user client
            version              print the version
            jar <jar>            run a jar file
            checknative [-a|-h]  check native hadoop and compression libraries availability
            distcp <srcurl> <desturl> copy file or directories recursively
            archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive
            classpath            prints the class path needed to get the
                                 Hadoop jar and the required libraries
            daemonlog            get/set the log level for each daemon
           or
            CLASSNAME            run the class named CLASSNAME
          
          Most commands print help when invoked w/o parameters.
          decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop checknative -h
          NativeLibraryChecker [-a|-h]
          decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop checknative
          12/12/21 15:07:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          Native library checking:
          hadoop: false
          zlib:   false
          snappy: false
          lz4:    false
          decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> echo $?
          1
          decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop checknative -a
          12/12/21 15:07:58 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
          Native library checking:
          hadoop: false
          zlib:   false
          snappy: false
          lz4:    false
          decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> echo $?
          1
          
          Show
          Binglin Chang added a comment - new version addressing my previous comments. As a single main class util, it's hard to add unit test. But I have done some simple manual test, here is the result. decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop Usage: hadoop [--config confdir] COMMAND where COMMAND is one of: fs run a generic filesystem user client version print the version jar <jar> run a jar file checknative [-a|-h] check native hadoop and compression libraries availability distcp <srcurl> <desturl> copy file or directories recursively archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive classpath prints the class path needed to get the Hadoop jar and the required libraries daemonlog get/set the log level for each daemon or CLASSNAME run the class named CLASSNAME Most commands print help when invoked w/o parameters. decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop checknative -h NativeLibraryChecker [-a|-h] decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop checknative 12/12/21 15:07:33 WARN util.NativeCodeLoader: Unable to load native -hadoop library for your platform... using builtin-java classes where applicable Native library checking: hadoop: false zlib: false snappy: false lz4: false decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> echo $? 1 decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> bin/hadoop checknative -a 12/12/21 15:07:58 WARN util.NativeCodeLoader: Unable to load native -hadoop library for your platform... using builtin-java classes where applicable Native library checking: hadoop: false zlib: false snappy: false lz4: false decster:~/projects/hadoop-trunk/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT> echo $? 1
          Hide
          Binglin Chang added a comment -

          Please add this functionality in a separate class instead of NativeCodeLoader.

          Please add this a command into src/main/bin/hadoop

          OK,I will put it in a new class, and add command to bin/hadoop

          Given that you are providing the availability status as output, exitcode of 1 only when native code is not available seems strange.

          I'm assuming people can use this in deploy scripts(tools) to check Native library availability, return -1 just a convenience for scripts, I can add a argument -a to check all native libraries(all available return 0) and default just check libhadoop(libhadoop available return 0).

          Because the snappy.jar file may not even be on the classpath, depending on the artifacts installed you need to

          I was not aware that I imported org.xerial.snappy.Snappy; it must be eclipse who mistakenly add it, actually it is not useful at all, I will remove it.

          It'll be more useful, if it can print the versions of the native libs as well. I've seen in the past it can pick up the wrong libs.

          I don't know how to achieve that, it seams a problem to introspect System.loadLibrary("hadoop"); to find out which file it actually load.

          Show
          Binglin Chang added a comment - Please add this functionality in a separate class instead of NativeCodeLoader. Please add this a command into src/main/bin/hadoop OK,I will put it in a new class, and add command to bin/hadoop Given that you are providing the availability status as output, exitcode of 1 only when native code is not available seems strange. I'm assuming people can use this in deploy scripts(tools) to check Native library availability, return -1 just a convenience for scripts, I can add a argument -a to check all native libraries(all available return 0) and default just check libhadoop(libhadoop available return 0). Because the snappy.jar file may not even be on the classpath, depending on the artifacts installed you need to I was not aware that I imported org.xerial.snappy.Snappy; it must be eclipse who mistakenly add it, actually it is not useful at all, I will remove it. It'll be more useful, if it can print the versions of the native libs as well. I've seen in the past it can pick up the wrong libs. I don't know how to achieve that, it seams a problem to introspect System.loadLibrary("hadoop"); to find out which file it actually load.
          Hide
          Luke Lu added a comment -

          It'll be more useful, if it can print the versions of the native libs as well. I've seen in the past it can pick up the wrong libs.

          Show
          Luke Lu added a comment - It'll be more useful, if it can print the versions of the native libs as well. I've seen in the past it can pick up the wrong libs.
          Hide
          Steve Loughran added a comment -

          Because the snappy.jar file may not even be on the classpath, depending on the artifacts installed you need to

          1. have the snappy test in a different class from NativeCodeLoader, so it wouldn't fail to load hadoop.lib just as snappy.jar was missing.
          2. split the zlib test from the snappy test
          Show
          Steve Loughran added a comment - Because the snappy.jar file may not even be on the classpath, depending on the artifacts installed you need to have the snappy test in a different class from NativeCodeLoader, so it wouldn't fail to load hadoop.lib just as snappy.jar was missing. split the zlib test from the snappy test
          Hide
          Suresh Srinivas added a comment -

          This is a useful utility. Comments:

          1. Please add this functionality in a separate class instead of NativeCodeLoader.
          2. Given that you are providing the availability status as output, exitcode of 1 only when native code is not available seems strange. Perhaps you should just always return 0.
          3. Please add this a command into src/main/bin/hadoop
          Show
          Suresh Srinivas added a comment - This is a useful utility. Comments: Please add this functionality in a separate class instead of NativeCodeLoader. Given that you are providing the availability status as output, exitcode of 1 only when native code is not available seems strange. Perhaps you should just always return 0. Please add this a command into src/main/bin/hadoop
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12561917/HADOOP-9162.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1914//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1914//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561917/HADOOP-9162.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1914//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1914//console This message is automatically generated.

            People

            • Assignee:
              Binglin Chang
              Reporter:
              Binglin Chang
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development