Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11927

Fix "undefined reference to dlopen" error when compiling libhadooppipes

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: build, native, tools
    • Labels:
      None
    • Environment:

      SUSE Linux Enterprise Server 11 SP3 (x86_64)

    • Target Version/s:

      Description

      When compile hadoop with native support, we encounter compile error that "undefined reference to `dlopen'" when link libcrypto. We'd better link libdl explicitly in CMakeList of hadoop-pips.

      1. HADOOP-11927-001.patch
        0.4 kB
        Xianyin Xin
      2. HADOOP-11927-002.patch
        0.6 kB
        Xianyin Xin

        Issue Links

          Activity

          Hide
          xinxianyin Xianyin Xin added a comment -

          Pre patch for review.

          Show
          xinxianyin Xianyin Xin added a comment - Pre patch for review.
          Hide
          trtrmitya Dmitry Sivachenko added a comment -

          Don't forget other OSes (like FreeBSD) which do not have libdl at all (dlopen() is in libc)

          Show
          trtrmitya Dmitry Sivachenko added a comment - Don't forget other OSes (like FreeBSD) which do not have libdl at all (dlopen() is in libc)
          Hide
          cnauroth Chris Nauroth added a comment -

          See HADOOP-8811 for a similar bug that was fixed in the hadoop-common native build.

          Show
          cnauroth Chris Nauroth added a comment - See HADOOP-8811 for a similar bug that was fixed in the hadoop-common native build.
          Hide
          xinxianyin Xianyin Xin added a comment -

          Thanks for your kind reminding Dmitry Sivachenko & Chris Nauroth.
          A simple fix. At present i don't have a FreeBSD env, but libc is linked automatically, so the fix is expected work on FreeBSD (if it can before the fix). However, i'll build a FreeBSD env and check it.

          Show
          xinxianyin Xianyin Xin added a comment - Thanks for your kind reminding Dmitry Sivachenko & Chris Nauroth . A simple fix. At present i don't have a FreeBSD env, but libc is linked automatically, so the fix is expected work on FreeBSD (if it can before the fix). However, i'll build a FreeBSD env and check it.
          Hide
          xinxianyin Xianyin Xin added a comment -

          A fix to avoid link libdl in FreeBSD like OSes.

          Show
          xinxianyin Xianyin Xin added a comment - A fix to avoid link libdl in FreeBSD like OSes.
          Hide
          xinxianyin Xianyin Xin added a comment -

          Check passed on FreeBSD 10.1, SUSE 11.3 SP3, CentOS 7.1 and Ubuntu 14.04.

          Show
          xinxianyin Xianyin Xin added a comment - Check passed on FreeBSD 10.1, SUSE 11.3 SP3, CentOS 7.1 and Ubuntu 14.04.
          Hide
          alanburlison Alan Burlison added a comment -

          On Solaris dlopen & friends are part of libc, no special treatment is required.

          Show
          alanburlison Alan Burlison added a comment - On Solaris dlopen & friends are part of libc, no special treatment is required.
          Hide
          cmccabe Colin P. McCabe added a comment -

          +1 pending jenkins

          Show
          cmccabe Colin P. McCabe added a comment - +1 pending jenkins
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 5m 29s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 7m 48s There were no new javac warning messages.
          +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 34s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
              15m 50s  



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12731055/HADOOP-11927-002.patch
          Optional Tests javac unit
          git revision trunk / e422e76
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6746/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6746/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 5m 29s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 48s There were no new javac warning messages. +1 release audit 0m 21s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.     15m 50s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12731055/HADOOP-11927-002.patch Optional Tests javac unit git revision trunk / e422e76 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6746/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6746/console This message was automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          committed to 2.8

          Show
          cmccabe Colin P. McCabe added a comment - committed to 2.8
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7895 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7895/)
          HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7895 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7895/ ) HADOOP-11927 . Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #205 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/205/)
          HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c)

          • hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #205 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/205/ ) HADOOP-11927 . Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c) hadoop-tools/hadoop-pipes/src/CMakeLists.txt hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #936 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/936/)
          HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #936 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/936/ ) HADOOP-11927 . Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2134 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2134/)
          HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2134 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2134/ ) HADOOP-11927 . Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/194/)
          HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #194 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/194/ ) HADOOP-11927 . Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #204 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/204/)
          HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c)

          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #204 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/204/ ) HADOOP-11927 . Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2152 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2152/)
          HADOOP-11927. Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c)

          • hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          • hadoop-common-project/hadoop-common/CHANGES.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2152 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2152/ ) HADOOP-11927 . Fix "undefined reference to dlopen" error when compiling libhadooppipes (Xianyin Xin via Colin P. McCabe) (cmccabe: rev f34638399abdca2e6d3bc5502508819dd747861c) hadoop-tools/hadoop-pipes/src/CMakeLists.txt hadoop-common-project/hadoop-common/CHANGES.txt
          Hide
          xinxianyin Xianyin Xin added a comment -

          Thanks Colin P. McCabe! And thanks Dmitry Sivachenko and Chris Nauroth for your valuable comments!

          Show
          xinxianyin Xianyin Xin added a comment - Thanks Colin P. McCabe ! And thanks Dmitry Sivachenko and Chris Nauroth for your valuable comments!

            People

            • Assignee:
              xinxianyin Xianyin Xin
              Reporter:
              xinxianyin Xianyin Xin
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development