Issue 122647 - Python linked against system OpenSSL
Python linked against system OpenSSL
Status: VERIFIED FIXED
Product: General
Classification: Code
Component: code
4.0.0-dev
All Unix, all
: P3 normal (vote)
: 4.0.0
Assigned To: hdu@apache.org
: release_blocker
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-30 04:48 UTC by Ariel Constenla-Haile
Modified: 2013-07-16 15:45 UTC (History)
4 users (show)

See Also:
Issue Type: DEFECT
Latest Confirmation on: ---
Developer Difficulty: ---
jsc: 4.0.0_release_blocker+


Attachments
suggested patch (3.67 KB, patch)
2013-07-01 16:10 UTC, hdu@apache.org
no flags Details | Diff
now with the new patch file (3.67 KB, patch)
2013-07-01 16:13 UTC, hdu@apache.org
no flags Details | Diff
without git-diff lines to help bugzilla (3.44 KB, patch)
2013-07-01 16:16 UTC, hdu@apache.org
no flags Details | Diff
without git-diff lines to help bugzilla (3.44 KB, patch)
2013-07-01 16:17 UTC, hdu@apache.org
no flags Details | Diff
Backtrace from the crash in Fedora 18, with the version built in CentOS 5 (22.34 KB, text/plain)
2013-07-05 08:05 UTC, Ariel Constenla-Haile
no flags Details
Backtrace from the crash in Fedora 18, with the version built in f18 (45.02 KB, text/plain)
2013-07-05 08:18 UTC, Ariel Constenla-Haile
no flags Details

Note You need to log in before you can comment on or make changes to this issue.
Description Ariel Constenla-Haile 2013-06-30 04:48:42 UTC
Python is linked against the OpenSSL libraries available on the system, even if configured with internal OpenSSL.

This breaks the mail service provider component, written in Python, as most SMTP services require SSL.

]$ ldd -v /opt/openoffice4/program/python-core-2.7.5/lib/lib-dynload/_ssl.so

        libssl.so.6 => not found
        libcrypto.so.6 => not found

]$ ldd -v /opt/openoffice4/program/python-core-2.7.5/lib/lib-dynload/_hashlib.so

        libssl.so.6 => not found
        libcrypto.so.6 => not found

Build log:

building '_ssl' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/usr/include -I/usr/local/include -I/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Include -I/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5 -c /build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_ssl.c -o build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_ssl.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_ssl.o -L/usr/local/lib -L. -lssl -lcrypto -lpython2.7 -o build/lib.linux-x86_64-2.7/_ssl.so
building '_hashlib' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/usr/include -I/usr/local/include -I/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Include -I/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5 -c /build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_hashopenssl.c -o build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_hashopenssl.o
gcc -pthread -shared build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_hashopenssl.o -L/usr/local/lib -L. -lssl -lcrypto -lpython2.7 -o build/lib.linux-x86_64-2.7/_hashlib.so
Comment 1 jsc 2013-07-01 06:43:08 UTC
set showstopper flag
Comment 2 hdu@apache.org 2013-07-01 16:10:59 UTC
Created attachment 80971 [details]
suggested patch

The suggested patch works by prefering include/library files in additional directories (which here means in solver) to the ones in the system default directories. Python's _ssl and _hashlib libraries then link against the the static ssl and crypto libraries as delivered the internal openssl module.
Comment 3 hdu@apache.org 2013-07-01 16:13:21 UTC
Created attachment 80972 [details]
now with the new patch file
Comment 4 hdu@apache.org 2013-07-01 16:16:15 UTC
Created attachment 80973 [details]
without git-diff lines to help bugzilla
Comment 5 hdu@apache.org 2013-07-01 16:17:43 UTC
Created attachment 80974 [details]
without git-diff lines to help bugzilla
Comment 6 SVN Robot 2013-07-02 08:38:40 UTC
"hdu" committed SVN 0 into trunk:
#i122647# link python against own libraries
Comment 7 hdu@apache.org 2013-07-02 08:42:19 UTC
Fixed by preferring headers and libraries (if available as requested by AOO's configure) to their system counterparts.
Comment 8 Ariel Constenla-Haile 2013-07-05 08:05:01 UTC
Created attachment 81003 [details]
Backtrace from the crash in Fedora 18, with the version built in CentOS 5

Testing Gmail SMTP server crashes in Fedora 18.

AOO400m3(Build:9702)  -  Rev. 1499347
2013-07-03 14:08:16 (Wed, 03 Jul 2013) - Linux x86_64
Comment 9 Ariel Constenla-Haile 2013-07-05 08:08:35 UTC
Reopening.
In CentOS 5 it does not crash and the mail service provider component works.
In Fedora 18 it crashed.

If someone has the latest and greatest Ubuntu/OpenSuse, please try.

Valid Gmail SMTP server configuration can be found in bug 119481 comment 1
Comment 10 Ariel Constenla-Haile 2013-07-05 08:18:42 UTC
Created attachment 81004 [details]
Backtrace from the crash in Fedora 18, with the version built in f18

Python compiled with debugging symbols.


Program received signal SIGSEGV, Segmentation fault.
0x000000385a2e0885 in EVP_PKEY_CTX_dup (pctx=0x4161f00) at pmeth_lib.c:310
310             if (!pctx->pmeth || !pctx->pmeth->copy)
(gdb) print *pctx
$1 = {
  pmeth = 0x1, 
  engine = 0x7fe03a251440 <PyString_Type>, 
  pkey = 0x4, 
  peerkey = 0xffffffffffffffff, 
  operation = 0, 
  data = 0x0, 
  app_data = 0x1, 
  pkey_gencb = 0x7fe03a251440 <PyString_Type>, 
  keygen_info = 0x6, 
  keygen_info_count = -1
}
(gdb) bt
#0  0x000000385a2e0885 in EVP_PKEY_CTX_dup (pctx=0x4161f00) at pmeth_lib.c:310
#1  0x000000385a2d2c7c in EVP_MD_CTX_copy_ex (out=0x415c908, in=0x7fe03867c1e0 <CONST_new_md5_ctx>) at digest.c:378
#2  0x000000385a2d2dd2 in EVP_MD_CTX_copy (out=<optimized out>, in=<optimized out>) at digest.c:329
#3  0x00007fe038406efb in EVPnew (name_obj=<optimized out>, digest=0x0, initial_ctx=0x7fe03867c1e0 <CONST_new_md5_ctx>, cp=0x0, len=0)
    at /build/aoo/src/playground/trunk/main/python/unxlngx6/misc/build/Python-2.7.5/Modules/_hashopenssl.c:436


For the backtrace, this is the system OpenSSL library. There is no pmeth_lib.c in the internal OpenSSL.
crypto/evp/pmeth_lib.c is present in Fedora 18's OpenSSL  (openssl-1.0.1e)
Comment 11 Ariel Constenla-Haile 2013-07-05 08:32:21 UTC
Strange. At compile time, it seems that now the openSSL library from the solver is taken:

Before:

gcc -pthread -shared 
build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_ssl.o 
-L/usr/local/lib 
-L. 
-lssl -lcrypto -lpython2.7 
-o build/lib.linux-x86_64-2.7/_ssl.so

Now:

gcc -pthread -shared 
build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_ssl.o 
-L/build/aoo/src/clean/svn-trunk/main/solver/400/unxlngx6.pro/lib 
-L/usr/local/lib 
-L. 
-lssl -lcrypto -lpython2.7 
-o build/lib.linux-x86_64-2.7/_ssl.so


And not linked to libssl.so.6/libcrypto.so.6:

~]$ ldd -v /opt/openoffice4/program/python-core-2.7.5/lib/lib-dynload/_ssl.so

linux-vdso.so.1 =>  (0x00007fff321fe000)
libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007fec4367a000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fec4345e000)
libc.so.6 => /lib64/libc.so.6 (0x00007fec430a5000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fec42ea1000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fec42c9e000)
libm.so.6 => /lib64/libm.so.6 (0x00007fec4299c000)
/lib64/ld-linux-x86-64.so.2 (0x000000384f800000)
Comment 12 hdu@apache.org 2013-07-05 09:34:27 UTC
As only the static ssl and crypto libraries are delivered to solver they get linked statically so they won't show up in the ldd result.

Why their dynamic counterparts from the system are loaded dynamically is a mystery. They can only be loaded via other libraries, but how can statically linked symbols ever lose out to indirectly dynamically loaded symbols? The symbols shown in the backtrace are clearly not the statically linked ones as their address is very different from the EVPnew address.
Comment 13 Ariel Constenla-Haile 2013-07-05 09:56:19 UTC
(In reply to hdu@apache.org from comment #12)
> Why their dynamic counterparts from the system are loaded dynamically is a
> mystery. They can only be loaded via other libraries, 

Indeed, simply invoking soffice (opens the start module) ends up loading openSSL libraries:

[ariel@localhost ~]$ pldd `pgrep soffice.bin` | grep crypto
/lib64/libcrypto.so.10
/lib64/libk5crypto.so.3
[ariel@localhost ~]$ pldd `pgrep soffice.bin` | grep ssl
/lib64/libssl.so.10


> but how can statically linked symbols ever lose out to indirectly dynamically loaded symbols? 

yes, strange.
Comment 14 hdu@apache.org 2013-07-05 15:18:06 UTC
Could reproduce the problem on RHEL6 after disabling the fix in 1. The call to EVP_MD_CTX_copy@plt seems to be resolved via symbol preemption. This can only happen for symbols with default visibility, but python does not yet do that [1]. Using the linker option "-Wl,--exclude-libs=ALL" might help. Investigating...

[1] http://bugs.python.org/issue1192789
Comment 15 SVN Robot 2013-07-05 16:23:07 UTC
"hdu" committed SVN 0 into trunk:
#i122647# prevent symbol preemption of openssl symbols in python libs
Comment 16 hdu@apache.org 2013-07-05 16:25:22 UTC
The linker option "exclude-libs ALL" solves the symbol preemption problem. Please verify.
Comment 17 hanya 2013-07-13 02:40:26 UTC
Verified with AOO4 RC on both Linux 32 bit and 64 bit (Xubuntu 12.04).
- Mail merge test settings works well with SSL connection.
- ./python -c "import ssl; print(ssl.OPENSSL_VERSION)" gave me "OpenSSL 0.9.8o 01 Jun 2010", 
  even with OpenSSL 1.0.1 14 Mar 2012 installed on the system.
Comment 18 hdu@apache.org 2013-07-16 15:45:32 UTC
(In reply to hanya from comment #17)
> Verified with AOO4 RC on both Linux 32 bit and 64 bit (Xubuntu 12.04).

Thanks for verifying the issue!