Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-10599

Prebuilt distributions (aka. pyarrow and libarrow-dev) should use the same ABI (with or without the DUAL abi)

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Critical
    • Resolution: Won't Fix
    • 0.17.0, 1.0.1, 2.0.0
    • None
    • C++, Python
    • None
    • Important

    Description

      I have observed that the python release (pyarrow) and c++ release (libarrow-dev for ubuntu) are built using the different GCC ABI.

      The former, pyarrow, builtin within the manylinux1 environment, using gcc-4.8, however the later's ABI has a `[cxx11]` tag. That blocks users to develop python C extensions that depends on libarrow-dev. For example, we have developed `lib` A in C++, which use arrow's `Arrow::Buffer` from libarrow-dev, and wrap it using things like `pybind11` to a python module `liba`. After building the `liba` on commodity Ubuntu (which could install libarrow-dev with apt-get), the user import both `liba` and `pyarrow` to the python's script, it won't work correctly due to the ABI confliction (especially when it comes to the string cases).

      I can see two options to make it works:

      1. build arrow's python package using static link, that the pyarrow won't contains so many shared libraries (libarrow.so, libarrow_python.so, etc.)
      2. distribute `libarrow-dev` with `-D_GLIBCXX_USE_CXX11_ABI=0`

      I'm also wondering if there's any technical issues that not distributing packages in different languages with the same ABI.

      Attachments

        Activity

          People

            Unassigned Unassigned
            sighingnow Tao He
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: