[IMPALA-3932] virtualenv does not build binary python packages with toolchain

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: Impala 2.6.0
• Fix Version/s: Impala 2.9.0
• Component/s: Infrastructure
• Labels: build python
• Target Version: Impala 2.9.0
• Assignee: Tim Armstrong
• Reporter: Matthew Jacobs
• Created: 28/Jul/16 22:46
• Updated: 07/Mar/17 05:59
• Resolved: 07/Mar/17 05:59

Description

The python virtualenv used by Impala installs a lot of packages, most of which are only python code. However, the sasl package (and maybe more) rely on C++ code which gets compiled at the time the package is installed by pip. Unfortunately the compilation does not use the Impala toolchain, so the compiled binaries reference the system libraries. On systems with a newer libstdc++ this is an issue when the sasl library is loaded.

E.g. on Ubuntu 16, any python code that imports sasl fails. This means we cannot even start the mini cluster because the code to start the Hive server imports sasl.

 --> Starting Hive Server and Metastore Service
Traceback (most recent call last):
  File "/data/impala-build/Impala/testdata/bin/wait-for-metastore.py", line 23, in <module>
    from tests.util.thrift_util import create_transport
  File "/data/impala-build/Impala/tests/util/thrift_util.py", line 20, in <module>
    import sasl
  File "build/bdist.linux-x86_64/egg/sasl/__init__.py", line 1, in <module>
  File "build/bdist.linux-x86_64/egg/sasl/saslwrapper.py", line 7, in <module>
  File "build/bdist.linux-x86_64/egg/_saslwrapper.py", line 7, in <module>
  File "build/bdist.linux-x86_64/egg/_saslwrapper.py", line 6, in __bootstrap__

ImportError: /tmp/toolchain-build/native-toolchain/build/gcc-4.9.2/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /root/.python-eggs/sasl-0.1.1-py2.7-linux-x86_64.egg-tmp/_saslwrapper.so)

Error in /data/impala-build/Impala/testdata/bin/run-hive-server.sh at line 54: impala-python ${CLUSTER_BIN}/wait-for-metastore.py --transport=${METASTORE_TRANSPORT}

Error in ./testdata/bin/run-all.sh at line 51: tee ${IMPALA_CLUSTER_LOGS_DIR}/run-hive-server.log

We need to either:
a) Figure out how to build python packages against our toolchain libraries. (This stackoverflow post may be helpful.)
b) Avoid setting LD_LIBRARY_PATH for the entire environment (IMPALA-3926)

I suspect (a) is worth doing now (we need this for Ubuntu 16), but (b) may be a better long term path.

Issue Links

• duplicates: IMPALA-4593 kudu-python is built with the system C++ compiler, which may not be ABI-compatible with the toolchain C++ compiler
• is related to: IMPALA-3926 Reconsider use of LD_LIBRARY_PATH for toolchain libraries
• is related to: IMPALA-4635 Reduce bootstrap time for Python virtualenv