Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8922

Link the native_mini_dfs test library with libdl, since IBM Java requires it

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: build
    • Labels:
      None
    • Environment:

      IBM Java RHEL7.1

    • Target Version/s:

      Description

      Building hadoop-hdfs-project with -Pnative option using IBM Java fails with the following error

      [exec] Linking C executable test_native_mini_dfs
      [exec] /usr/bin/cmake -E cmake_link_script CMakeFiles/test_native_mini_dfs.dir/link.txt --verbose=1
      [exec] /usr/bin/cc -g -Wall -O2 -D_REENTRANT -D_GNU_SOURCE -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -fvisibility=hidden CMakeFiles/test_native_mini_dfs.dir/main/native/libhdfs/test_native_mini_dfs.c.o -o test_native_mini_dfs -rdynamic libnative_mini_dfs.a /home/ayappan/ibm-java-ppc64le-71/jre/lib/ppc64le/classic/libjvm.so -lpthread -Wl,-rpath,/home/ayappan/ibm-java-ppc64le-71/jre/lib/ppc64le/classic
      [exec] make[2]: Leaving directory `/home/ayappan/hadoop_2.7.1_new/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/native'
      [exec] make[1]: Leaving directory `/home/ayappan/hadoop_2.7.1_new/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/native'
      [exec] /home/ayappan/ibm-java-ppc64le-71/jre/lib/ppc64le/classic/libjvm.so: undefined reference to `dlopen'
      [exec] /home/ayappan/ibm-java-ppc64le-71/jre/lib/ppc64le/classic/libjvm.so: undefined reference to `dlclose'
      [exec] /home/ayappan/ibm-java-ppc64le-71/jre/lib/ppc64le/classic/libjvm.so: undefined reference to `dlerror'
      [exec] /home/ayappan/ibm-java-ppc64le-71/jre/lib/ppc64le/classic/libjvm.so: undefined reference to `dlsym'
      [exec] /home/ayappan/ibm-java-ppc64le-71/jre/lib/ppc64le/classic/libjvm.so: undefined reference to `dladdr'
      [exec] collect2: error: ld returned 1 exit status
      [exec] make[2]: *** [test_native_mini_dfs] Error 1
      [exec] make[1]: *** [CMakeFiles/test_native_mini_dfs.dir/all] Error 2
      [exec] make: *** [all] Error 2

      It seems like the IBM jvm requires libdl for linking in native_mini_dfs.

        Activity

        Hide
        Ayappan Ayappan added a comment -

        seems like already fixed in the latest development branch

        Show
        Ayappan Ayappan added a comment - seems like already fixed in the latest development branch
        Hide
        Ayappan Ayappan added a comment -

        Sorry, my mistake for the above comments. This problem is still there in the development branch.
        I am attaching a patch here for the same.

        Show
        Ayappan Ayappan added a comment - Sorry, my mistake for the above comments. This problem is still there in the development branch. I am attaching a patch here for the same.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 5m 44s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 13s There were no new javac warning messages.
        +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 25s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 native 1m 1s Pre-build of native portion
        -1 hdfs tests 176m 59s Tests failed in hadoop-hdfs.
            194m 21s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
        Timed out tests org.apache.hadoop.cli.TestHDFSCLI



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12751299/HDFS-8922.patch
        Optional Tests javac unit
        git revision trunk / f61120d
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12043/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12043/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12043/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 44s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 13s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 25s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 native 1m 1s Pre-build of native portion -1 hdfs tests 176m 59s Tests failed in hadoop-hdfs.     194m 21s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12751299/HDFS-8922.patch Optional Tests javac unit git revision trunk / f61120d hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12043/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12043/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12043/console This message was automatically generated.
        Hide
        cmccabe Colin P. McCabe added a comment -

        +1

        Show
        cmccabe Colin P. McCabe added a comment - +1
        Hide
        Ayappan Ayappan added a comment -

        Test Failure seems to be unrelated to this patch.

        Show
        Ayappan Ayappan added a comment - Test Failure seems to be unrelated to this patch.
        Hide
        cmccabe Colin P. McCabe added a comment -

        I have to admit, I do find it kind of strange that IBM's libjvm.so doesn't depend on libdl.so itself, if it needs to make use of that library. However, given that this only affects test programs, and it seems like a trivial fix, let's just do it. I agree that the test failures are unrelated.

        Show
        cmccabe Colin P. McCabe added a comment - I have to admit, I do find it kind of strange that IBM's libjvm.so doesn't depend on libdl.so itself, if it needs to make use of that library. However, given that this only affects test programs, and it seems like a trivial fix, let's just do it. I agree that the test failures are unrelated.
        Hide
        cmccabe Colin P. McCabe added a comment -

        committed to 2.8, thanks

        Show
        cmccabe Colin P. McCabe added a comment - committed to 2.8, thanks
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8330 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8330/)
        HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083)

        • hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8330 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8330/ ) HDFS-8922 . Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083) hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/291/)
        HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/291/ ) HDFS-8922 . Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #1024 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1024/)
        HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083)

        • hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1024 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1024/ ) HDFS-8922 . Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083) hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #294 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/294/)
        HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083)

        • hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #294 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/294/ ) HDFS-8922 . Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083) hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2240 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2240/)
        HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083)

        • hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2240 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2240/ ) HDFS-8922 . Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083) hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #283 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/283/)
        HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #283 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/283/ ) HDFS-8922 . Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #2221 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2221/)
        HDFS-8922. Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083)

        • hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2221 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2221/ ) HDFS-8922 . Link the native_mini_dfs test library with libdl, since IBM Java requires it (Ayappan via Colin P. McCabe) (cmccabe: rev 7642f64c24961d2b4772591a0957e2699162a083) hadoop-hdfs-project/hadoop-hdfs/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            Ayappan Ayappan
            Reporter:
            Ayappan Ayappan
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development