Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3753

Tests don't run with native libraries

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: build, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Test execution when run with the native flag and native libraries have been built don't actually use the native libs because NativeCodeLoader is unable to load native-hadoop. Eg run mvn compile -Pnative then mvn -Dtest=TestSeekBug test -Pnative and check the test output. This is because the test's java.library.path is looking for the lib in hdfs (
      hadoop-hdfs-project/hadoop-hdfs/target/native/target/usr/local/lib) however the native lib lives in common. I confirmed copying the lib to the appropriate directory fixes things. We need to update the java.library.path for test execution to include the common lib dir. This may be an issue with MR as well.

      1. HDFS-3753.001.patch
        7 kB
        Colin Patrick McCabe
      2. HDFS-3753.002.patch
        9 kB
        Colin Patrick McCabe

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          54d 5h 40m 1 Colin Patrick McCabe 26/Sep/12 00:00
          Patch Available Patch Available Resolved Resolved
          6d 22h 15m 1 Eli Collins 02/Oct/12 22:15
          Resolved Resolved Closed Closed
          135d 15h 56m 1 Arun C Murthy 15/Feb/13 13:12
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1216/)
          Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003. (Revision 1393830)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1216 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1216/ ) Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003 . (Revision 1393830) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1185/)
          Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003. (Revision 1393830)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1185 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1185/ ) Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003 . (Revision 1393830) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2831 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2831/)
          Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003. (Revision 1393830)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2831 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2831/ ) Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003 . (Revision 1393830) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2808 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2808/)
          Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003. (Revision 1393830)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2808 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2808/ ) Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003 . (Revision 1393830) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2870 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2870/)
          Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003. (Revision 1393830)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830
          Files :

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2870 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2870/ ) Amend HDFS-3753 to ignore TestHdfsNativeCodeLoader. To be fixed in HDFS-4003 . (Revision 1393830) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393830 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          Hide
          Eli Collins added a comment -

          Filed HDFS-4003.

          Show
          Eli Collins added a comment - Filed HDFS-4003 .
          Eli Collins made changes -
          Link This issue relates to HDFS-4003 [ HDFS-4003 ]
          Hide
          Eli Collins added a comment -

          The test in HDFS-4000 failed again, the issue here is that test-patch does a clean several times so unless the patch modifies common the native library is not present. I'll ammend this patch to add an @Ignore TestHdfsNativeCodeLoader and we'll do a follow on change to remove the @Ignore and modify test-patch to run the common native build before running the Hdfs tests.

          Show
          Eli Collins added a comment - The test in HDFS-4000 failed again, the issue here is that test-patch does a clean several times so unless the patch modifies common the native library is not present. I'll ammend this patch to add an @Ignore TestHdfsNativeCodeLoader and we'll do a follow on change to remove the @Ignore and modify test-patch to run the common native build before running the Hdfs tests.
          Hide
          Colin Patrick McCabe added a comment -

          Thanks, Eli. These kind of environment setup issues are exactly what the test was meant to catch, and it looks like they have.

          Show
          Colin Patrick McCabe added a comment - Thanks, Eli. These kind of environment setup issues are exactly what the test was meant to catch, and it looks like they have.
          Hide
          Eli Collins added a comment -

          I kicked the Pre Commit job to re-run HDFS-4000 to make sure this works.

          Show
          Eli Collins added a comment - I kicked the Pre Commit job to re-run HDFS-4000 to make sure this works.
          Hide
          Eli Collins added a comment -

          Looks like TestNativeCodeLoader is failing because it can't find libhadoop, because the common native build was not run. I've updated the HDFS pre commit job to build the common native bits first.

          The reason we didn't catch this here is that the Pre Commit job doesn't pick up on the test-patch.sh changes from the uploaded patch, I filed HADOOP-8875 for that.

          Show
          Eli Collins added a comment - Looks like TestNativeCodeLoader is failing because it can't find libhadoop, because the common native build was not run. I've updated the HDFS pre commit job to build the common native bits first. The reason we didn't catch this here is that the Pre Commit job doesn't pick up on the test-patch.sh changes from the uploaded patch, I filed HADOOP-8875 for that.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1215 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1215/)
          HDFS-3753. Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113
          Files :

          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1215 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1215/ ) HDFS-3753 . Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113 Files : /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1184 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1184/)
          HDFS-3753. Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113
          Files :

          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1184 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1184/ ) HDFS-3753 . Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113 Files : /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Eli Collins added a comment -

          Looks like TestNativeCodeLoader is failing in some other runs, eg HDFS-3995.

          Show
          Eli Collins added a comment - Looks like TestNativeCodeLoader is failing in some other runs, eg HDFS-3995 .
          Hide
          Eli Collins added a comment -

          TestParallelLocalRead may be related to this. Filed HDFS-4000.

          Show
          Eli Collins added a comment - TestParallelLocalRead may be related to this. Filed HDFS-4000 .
          Eli Collins made changes -
          Link This issue relates to HDFS-4000 [ HDFS-4000 ]
          Eli Collins made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Target Version/s 2.0.2-alpha [ 12322472 ]
          Fix Version/s 2.0.3-alpha [ 12323274 ]
          Resolution Fixed [ 1 ]
          Hide
          Eli Collins added a comment -

          I've committed this and merged to branch-2. Thanks Colin!

          Show
          Eli Collins added a comment - I've committed this and merged to branch-2. Thanks Colin!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2825 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2825/)
          HDFS-3753. Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113)

          Result = FAILURE
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113
          Files :

          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2825 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2825/ ) HDFS-3753 . Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113 Files : /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Todd Lipcon added a comment -

          Mark this as resolved? Or is it awaiting commit to more branches?

          Show
          Todd Lipcon added a comment - Mark this as resolved? Or is it awaiting commit to more branches?
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2864 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2864/)
          HDFS-3753. Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113
          Files :

          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2864 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2864/ ) HDFS-3753 . Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113 Files : /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2802 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2802/)
          HDFS-3753. Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113)

          Result = SUCCESS
          eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113
          Files :

          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2802 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2802/ ) HDFS-3753 . Tests don't run with native libraries. Contributed by Colin Patrick McCabe (Revision 1393113) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1393113 Files : /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/TestHdfsNativeCodeLoader.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestShortCircuitLocalRead.java /hadoop/common/trunk/hadoop-project/pom.xml
          Hide
          Eli Collins added a comment -

          +1 looks great

          Nit: LD_LIBRARY_PATH shouldn't be indented more, I'll fix that when I commit

          Show
          Eli Collins added a comment - +1 looks great Nit: LD_LIBRARY_PATH shouldn't be indented more, I'll fix that when I commit
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12547248/HDFS-3753.002.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3249//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3249//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12547248/HDFS-3753.002.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 4 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3249//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3249//console This message is automatically generated.
          Colin Patrick McCabe made changes -
          Attachment HDFS-3753.002.patch [ 12546750 ]
          Colin Patrick McCabe made changes -
          Attachment HDFS-3753.002.patch [ 12547248 ]
          Hide
          Colin Patrick McCabe added a comment -

          resubmitting patch 002 so that jenkins runs it

          Show
          Colin Patrick McCabe added a comment - resubmitting patch 002 so that jenkins runs it
          Eli Collins made changes -
          Assignee Colin Patrick McCabe [ cmccabe ]
          Colin Patrick McCabe made changes -
          Attachment HDFS-3753.002.patch [ 12546750 ]
          Hide
          Colin Patrick McCabe added a comment -

          TestShortCircuitLocalRead is currently broken in the presence of libhadoop.so. This patch fixes TestShortCircuitLocalRead for this case.

          Show
          Colin Patrick McCabe added a comment - TestShortCircuitLocalRead is currently broken in the presence of libhadoop.so. This patch fixes TestShortCircuitLocalRead for this case.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12546607/HDFS-3753.001.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestShortCircuitLocalRead

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3237//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3237//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12546607/HDFS-3753.001.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestShortCircuitLocalRead +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3237//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3237//console This message is automatically generated.
          Colin Patrick McCabe made changes -
          Attachment HDFS-3753.001.patch [ 12546607 ]
          Colin Patrick McCabe made changes -
          Attachment HDFS-3975.001.patch [ 12546606 ]
          Colin Patrick McCabe made changes -
          Attachment HDFS-3753.001.patch [ 12546605 ]
          Colin Patrick McCabe made changes -
          Attachment HDFS-3975.001.patch [ 12546606 ]
          Colin Patrick McCabe made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Colin Patrick McCabe made changes -
          Attachment HDFS-3753.001.patch [ 12546605 ]
          Hide
          Colin Patrick McCabe added a comment -

          This patch adds the hadoop-common native library directory to LD_LIBRARY_PATH, so that the junit tests of other projects, such as hadoop-hdfs-project, will be run with libhadoop.so if -Pnative was specified. This ensures that we get test coverage of HDFS running with libhadoop.so present.

          This patch also adds two new junit tests, TestNativeCodeLoader and TestHdfsNativeCodeLoader. If the system property -Drequire.test.libhadoop is set, these tests will ensure that libhadoop.so is actually being tested. This is important for repeatable builds, and to prevent this bug from cropping up again-- for example if libhadoop.so is moved.

          Finally, this adds -Drequire.test.libhadoop to test-patch.sh. This ensures that any regressions that lead to libhadoop.so not being tested are spotted by Jenkins.

          Show
          Colin Patrick McCabe added a comment - This patch adds the hadoop-common native library directory to LD_LIBRARY_PATH, so that the junit tests of other projects, such as hadoop-hdfs-project, will be run with libhadoop.so if -Pnative was specified. This ensures that we get test coverage of HDFS running with libhadoop.so present. This patch also adds two new junit tests, TestNativeCodeLoader and TestHdfsNativeCodeLoader. If the system property -Drequire.test.libhadoop is set, these tests will ensure that libhadoop.so is actually being tested. This is important for repeatable builds, and to prevent this bug from cropping up again-- for example if libhadoop.so is moved. Finally, this adds -Drequire.test.libhadoop to test-patch.sh. This ensures that any regressions that lead to libhadoop.so not being tested are spotted by Jenkins.
          Jeff Hammerbacher made changes -
          Link This issue relates to HDFS-3754 [ HDFS-3754 ]
          Colin Patrick McCabe made changes -
          Field Original Value New Value
          Link This issue relates to HADOOP-8648 [ HADOOP-8648 ]
          Eli Collins created issue -

            People

            • Assignee:
              Colin Patrick McCabe
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development