Hadoop Common
  1. Hadoop Common
  2. HADOOP-8642

io.native.lib.available only controls zlib

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: native
    • Labels:
      None

      Description

      Per core-default.xml io.native.lib.available indicates "Should native hadoop libraries, if present, be used" however it looks like it only affects zlib. Since we always load the native library this means we may use native libraries even if io.native.lib.available is set to false.

      Let's make the flag to work as advertised - rather than always load the native hadoop library we only attempt to load the library (and report that native is available) if this flag is set. Since io.native.lib.available defaults to true the default behavior should remain unchanged (except that now we wont actually try to load the library if this flag is disabled).

      1. HADOOP-8642.patch
        4 kB
        Akira AJISAKA
      2. HADOOP-8642.2.patch
        4 kB
        Akira AJISAKA

        Activity

        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12622852/HADOOP-8642.2.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3432//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3432//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12622852/HADOOP-8642.2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3432//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3432//console This message is automatically generated.
        Hide
        Akira AJISAKA added a comment -

        Renewed the patch to pass the failed tests.

        Show
        Akira AJISAKA added a comment - Renewed the patch to pass the failed tests.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12622832/HADOOP-8642.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.io.compress.TestCodec
        org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3431//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3431//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12622832/HADOOP-8642.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.io.compress.TestCodec org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3431//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3431//console This message is automatically generated.
        Hide
        Akira AJISAKA added a comment -

        Attaching a patch.
        Now NativeCodeLoader always try to load native library in static method.
        The patch will check configuration in the method. If io.native.lib.available is false, NativeCodeLoader never try to load native library.

        Show
        Akira AJISAKA added a comment - Attaching a patch. Now NativeCodeLoader always try to load native library in static method. The patch will check configuration in the method. If io.native.lib.available is false, NativeCodeLoader never try to load native library.
        Hide
        jay vyas added a comment -

        this is a very important feature. Just spent a few hours trying to figure out why NativeIO was still being invoked by RawLocalFileSystem even though i had turned io.native.lib.available off.

        Show
        jay vyas added a comment - this is a very important feature. Just spent a few hours trying to figure out why NativeIO was still being invoked by RawLocalFileSystem even though i had turned io.native.lib.available off.

          People

          • Assignee:
            Akira AJISAKA
            Reporter:
            Eli Collins
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:

              Development