Issue 122647

set showstopper flag

Created attachment 80971 [details]
suggested patch

The suggested patch works by prefering include/library files in additional directories (which here means in solver) to the ones in the system default directories. Python's _ssl and _hashlib libraries then link against the the static ssl and crypto libraries as delivered the internal openssl module.

Created attachment 80972 [details]
now with the new patch file

Created attachment 80973 [details]
without git-diff lines to help bugzilla

Created attachment 80974 [details]
without git-diff lines to help bugzilla

"hdu" committed SVN revision 1498829 into trunk:
#i122647# link python against own libraries

Fixed by preferring headers and libraries (if available as requested by AOO's configure) to their system counterparts.

Created attachment 81003 [details]
Backtrace from the crash in Fedora 18, with the version built in CentOS 5

Testing Gmail SMTP server crashes in Fedora 18.

AOO400m3(Build:9702)  -  Rev. 1499347
2013-07-03 14:08:16 (Wed, 03 Jul 2013) - Linux x86_64

Reopening.
In CentOS 5 it does not crash and the mail service provider component works.
In Fedora 18 it crashed.

If someone has the latest and greatest Ubuntu/OpenSuse, please try.

Valid Gmail SMTP server configuration can be found in bug 119481 comment 1

Created attachment 81004 [details]
Backtrace from the crash in Fedora 18, with the version built in f18

Python compiled with debugging symbols.


Program received signal SIGSEGV, Segmentation fault.
0x000000385a2e0885 in EVP_PKEY_CTX_dup (pctx=0x4161f00) at pmeth_lib.c:310
310             if (!pctx->pmeth || !pctx->pmeth->copy)
(gdb) print *pctx
$1 = {
  pmeth = 0x1, 
  engine = 0x7fe03a251440 <PyString_Type>, 
  pkey = 0x4, 
  peerkey = 0xffffffffffffffff, 
  operation = 0, 
  data = 0x0, 
  app_data = 0x1, 
  pkey_gencb = 0x7fe03a251440 <PyString_Type>, 
  keygen_info = 0x6, 
  keygen_info_count = -1
}
(gdb) bt
#0  0x000000385a2e0885 in EVP_PKEY_CTX_dup (pctx=0x4161f00) at pmeth_lib.c:310
#1  0x000000385a2d2c7c in EVP_MD_CTX_copy_ex (out=0x415c908, in=0x7fe03867c1e0 <CONST_new_md5_ctx>) at digest.c:378
#2  0x000000385a2d2dd2 in EVP_MD_CTX_copy (out=<optimized out>, in=<optimized out>) at digest.c:329
#3  0x00007fe038406efb in EVPnew (name_obj=<optimized out>, digest=0x0, initial_ctx=0x7fe03867c1e0 <CONST_new_md5_ctx>, cp=0x0, len=0)
    at /build/aoo/src/playground/trunk/main/python/unxlngx6/misc/build/Python-2.7.5/Modules/_hashopenssl.c:436


For the backtrace, this is the system OpenSSL library. There is no pmeth_lib.c in the internal OpenSSL.
crypto/evp/pmeth_lib.c is present in Fedora 18's OpenSSL  (openssl-1.0.1e)

Strange. At compile time, it seems that now the openSSL library from the solver is taken:

Before:

gcc -pthread -shared 
build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_ssl.o 
-L/usr/local/lib 
-L. 
-lssl -lcrypto -lpython2.7 
-o build/lib.linux-x86_64-2.7/_ssl.so

Now:

gcc -pthread -shared 
build/temp.linux-x86_64-2.7/build/aoo/src/clean/svn-trunk/main/python/unxlngx6.pro/misc/build/Python-2.7.5/Modules/_ssl.o 
-L/build/aoo/src/clean/svn-trunk/main/solver/400/unxlngx6.pro/lib 
-L/usr/local/lib 
-L. 
-lssl -lcrypto -lpython2.7 
-o build/lib.linux-x86_64-2.7/_ssl.so


And not linked to libssl.so.6/libcrypto.so.6:

~]$ ldd -v /opt/openoffice4/program/python-core-2.7.5/lib/lib-dynload/_ssl.so

linux-vdso.so.1 =>  (0x00007fff321fe000)
libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007fec4367a000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fec4345e000)
libc.so.6 => /lib64/libc.so.6 (0x00007fec430a5000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fec42ea1000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fec42c9e000)
libm.so.6 => /lib64/libm.so.6 (0x00007fec4299c000)
/lib64/ld-linux-x86-64.so.2 (0x000000384f800000)

As only the static ssl and crypto libraries are delivered to solver they get linked statically so they won't show up in the ldd result.

Why their dynamic counterparts from the system are loaded dynamically is a mystery. They can only be loaded via other libraries, but how can statically linked symbols ever lose out to indirectly dynamically loaded symbols? The symbols shown in the backtrace are clearly not the statically linked ones as their address is very different from the EVPnew address.

(In reply to hdu@apache.org from comment #12)
> Why their dynamic counterparts from the system are loaded dynamically is a
> mystery. They can only be loaded via other libraries, 

Indeed, simply invoking soffice (opens the start module) ends up loading openSSL libraries:

[ariel@localhost ~]$ pldd `pgrep soffice.bin` | grep crypto
/lib64/libcrypto.so.10
/lib64/libk5crypto.so.3
[ariel@localhost ~]$ pldd `pgrep soffice.bin` | grep ssl
/lib64/libssl.so.10


> but how can statically linked symbols ever lose out to indirectly dynamically loaded symbols? 

yes, strange.

Could reproduce the problem on RHEL6 after disabling the fix in revision 1498507. The call to EVP_MD_CTX_copy@plt seems to be resolved via symbol preemption. This can only happen for symbols with default visibility, but python does not yet do that [1]. Using the linker option "-Wl,--exclude-libs=ALL" might help. Investigating...

[1] http://bugs.python.org/issue1192789

"hdu" committed SVN revision 1500058 into trunk:
#i122647# prevent symbol preemption of openssl symbols in python libs

The linker option "exclude-libs ALL" solves the symbol preemption problem. Please verify.

Verified with AOO4 RC on both Linux 32 bit and 64 bit (Xubuntu 12.04).
- Mail merge test settings works well with SSL connection.
- ./python -c "import ssl; print(ssl.OPENSSL_VERSION)" gave me "OpenSSL 0.9.8o 01 Jun 2010", 
  even with OpenSSL 1.0.1 14 Mar 2012 installed on the system.

(In reply to hanya from comment #17)
> Verified with AOO4 RC on both Linux 32 bit and 64 bit (Xubuntu 12.04).

Thanks for verifying the issue!