Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1368

libarrow.a is not linked against boost libraries when compiled with -DARROW_BOOST_USE_SHARED=off

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: C++
    • Labels:
      None
    • Environment:
      Ubuntu 16.04

      Description

      When I build arrow with -DARROW_BOOST_USE_SHARED=off and then inspect libarrow.a with nm -g libarrow.a, some boost symbols are undefined.

      The problem can be reproduced on Ubuntu 16.04 as follows.

      First compile boost with -fPIC.

      cd ~
      wget --no-check-certificate http://downloads.sourceforge.net/project/boost/boost/1.60.0/boost_1_60_0.tar.gz -O ~/boost_1_60_0.tar.gz
      tar xf boost_1_60_0.tar.gz
      cd boost_1_60_0/
      ./bootstrap.sh
      ./bjam cxxflags=-fPIC cflags=-fPIC --prefix=../boost --with-filesystem --with-date_time --with-system --with-regex install
      cd ..
      

      Then compile Arrow.

      cd ~
      git clone https://github.com/apache/arrow
      mkdir -p ~/arrow/cpp/build
      cd ~/arrow/cpp/build
      BOOST_ROOT=~/boost \
      cmake -DCMAKE_BUILD_TYPE=Release \
            -DCMAKE_C_FLAGS="-g -O3" \
            -DCMAKE_CXX_FLAGS="-g -O3" \
            -DARROW_BUILD_TESTS=off \
            -DARROW_HDFS=on \
            -DARROW_BOOST_USE_SHARED=off \
            -DARROW_PYTHON=on \
            -DARROW_PLASMA=on \
            -DPLASMA_PYTHON=on \
            -DARROW_JEMALLOC=off \
            -DARROW_WITH_BROTLI=off \
            -DARROW_WITH_LZ4=off \
            -DARROW_WITH_SNAPPY=off \
            -DARROW_WITH_ZLIB=off \
            -DARROW_WITH_ZSTD=off \
            ..
      
      make VERBOSE=1 -j8
      

      The cmake command finds the recently compiled boost and prints the following.

      -- [ /usr/share/cmake-3.5/Modules/FindBoost.cmake:1516 ] Boost_FOUND = 1
      -- Boost version: 1.60.0
      -- Found the following Boost libraries:
      --   system
      --   filesystem
      -- Boost include dir: /home/ubuntu/boost/include
      -- Boost libraries: /home/ubuntu/boost/lib/libboost_system.a/home/ubuntu/boost/lib/libboost_filesystem.a
      Added static library dependency boost_system: /home/ubuntu/boost/lib/libboost_system.a
      Added static library dependency boost_filesystem: /home/ubuntu/boost/lib/libboost_filesystem.a
      

      Compilation does not appear to link libarrow.a against the boost libraries (though libarrow.so is handled properly).

      For libarrow.a (not linked against boost)

      /usr/bin/ar qc release/libarrow.a  CMakeFiles/arrow_objlib.dir/src/arrow/array.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/buffer.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/builder.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/compare.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/memory_pool.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/pretty_print.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/status.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/table.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/tensor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/type.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/visitor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/file.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/interfaces.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/memory.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/bit-util.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/compression.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/cpu-info.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/decimal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/key_value_metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/feather.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/reader.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/writer.cc.o
      

      For libarrow.so (linked against boost)

      /usr/bin/c++  -fPIC -g -O3 -O3 -DNDEBUG -Wall -std=c++11 -msse3  -O3 -DNDEBUG -Wl,--version-script=/home/ubuntu/arrow/cpp/src/arrow/symbols.map -shared -Wl,-soname,libarrow.so.0 -o release/libarrow.so.0.0.0 CMakeFiles/arrow_objlib.dir/src/arrow/array.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/buffer.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/builder.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/compare.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/memory_pool.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/pretty_print.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/status.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/table.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/tensor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/type.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/visitor.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/file.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/interfaces.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/memory.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/bit-util.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/compression.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/cpu-info.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/decimal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/util/key_value_metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/io/hdfs-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/feather.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/json-internal.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/metadata.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/reader.cc.o CMakeFiles/arrow_objlib.dir/src/arrow/ipc/writer.cc.o -ldl -lpthread /home/ubuntu/boost/lib/libboost_system.a /home/ubuntu/boost/lib/libboost_filesystem.a 
      

      Then nm -g release/libarrow.a contains the following line (for example).

                       U _ZN5boost6system15system_categoryEv
      

      The file arrow/cpp/CMakeLists.txt contains the block

      ADD_ARROW_LIB(arrow
        SOURCES ${ARROW_ALL_SRCS}
        DEPENDENCIES arrow_dependencies
        SHARED_LINK_FLAGS ${ARROW_SHARED_LINK_FLAGS}
        SHARED_LINK_LIBS ${ARROW_LINK_LIBS}
        SHARED_PRIVATE_LINK_LIBS ${ARROW_SHARED_PRIVATE_LINK_LIBS}
        STATIC_LINK_LIBS ${ARROW_STATIC_LINK_LIBS}
        STATIC_PRIVATE_LINK_LIBS ${ARROW_STATIC_PRIVATE_LINK_LIBS}
      )
      

      Printing ARROW_STATIC_PRIVATE_LINK_LIBS shows that the variable equals boost_system_static;boost_filesystem_static, which seems right.

      Let me know if you have any thoughts about this.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              robertnishihara Robert Nishihara
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: