The httpd build is failing with a segfault on Solaris 7. This occurs at the same place on two different Sun servers. A Sun Netra and a Sun 250. This is occuring both using gcc and Sun CC. Both of these servers previously built httpd 2.0.55 and 2.0.54 successfully and are currently using those versions for production. Here are the relevant lines output when the make fails: /usr/local/kinetic/split/httpd-2.0.58/srclib/apr/libtool --silent --mode=link gcc -g -O2 -pthreads -DSOLARIS2=7 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -DAP_HAVE_DESIGNATED_INITIALIZER -I/usr/local/kinetic/split/httpd-2.0.58/srclib/apr/include -I/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/include -I/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib -I. -I/usr/local/kinetic/split/httpd-2.0.58/os/unix -I/usr/local/kinetic/split/httpd-2.0.58/server/mpm/worker -I/usr/local/kinetic/split/httpd-2.0.58/modules/http -I/usr/local/kinetic/split/httpd-2.0.58/modules/filters -I/usr/local/kinetic/split/httpd-2.0.58/modules/proxy -I/usr/local/kinetic/split/httpd-2.0.58/include -I/usr/local/kinetic/split/httpd-2.0.58/modules/generators -I/usr/local/kinetic/ssl/include/openssl -I/usr/local/kinetic/ssl/include -I/usr/local/kinetic/split/httpd-2.0.58/modules/dav/main -export-dynamic -L/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib -L/usr/local/kinetic/ssl/lib -R/usr/local/kinetic/ssl/lib -o httpd modules.lo modules/aaa/mod_access.la modules/aaa/mod_auth.la modules/filters/mod_include.la modules/loggers/mod_log_config.la modules/metadata/mod_env.la modules/metadata/mod_usertrack.la modules/metadata/mod_setenvif.la modules/http/mod_http.la modules/http/mod_mime.la modules/generators/mod_status.la modules/generators/mod_autoindex.la modules/generators/mod_asis.la modules/generators/mod_cgid.la modules/generators/mod_cgi.la modules/mappers/mod_negotiation.la modules/mappers/mod_dir.la modules/mappers/mod_imap.la modules/mappers/mod_actions.la modules/mappers/mod_userdir.la modules/mappers/mod_alias.la modules/mappers/mod_so.la server/mpm/worker/libworker.la server/libmain.la os/unix/libos.la -lssl -lcrypto /usr/local/kinetic/split/httpd-2.0.58/srclib/pcre/libpcre.la /usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/libaprutil-0.la /usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib/libexpat.la -liconv /usr/local/kinetic/split/httpd-2.0.58/srclib/apr/libapr-0.la -lrt -lm -lsocket -lnsl -lresolv -lpthread -ldl /usr/local/kinetic/split/httpd-2.0.58/srclib/apr/libtool[4767]: : is not an identifier make[1]: *** [httpd] Segmentation Fault (core dumped) make[1]: Leaving directory `/usr/local/kinetic/split/httpd-2.0.58' make: *** [all-recursive] Error 1 Here is the relevant output from truss when the make fails: 4394/1: Received signal #18, SIGCLD, in read() [caught] 4394/1: siginfo: SIGCLD CLD_EXITED pid=5854 status=0x0000 4394/1: read(3, 0xFFBECE80, 1024) Err#4 EINTR 4394/1: setcontext(0xFFBEC4A8) 4394/1: sigaction(SIGCLD, 0xFFBECA38, 0xFFBECAB8) = 0 4394/1: waitid(P_ALL, 0, 0xFFBECA78, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 4394/1: waitid(P_ALL, 0, 0xFFBECA78, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) Err#10 ECHILD 4394/1: sigaction(SIGCLD, 0xFFBECA38, 0xFFBECAB8) = 0 4394/1: sigaction(SIGCLD, 0xFFBEC5F8, 0xFFBEC678) = 0 4394/1: waitid(P_ALL, 0, 0xFFBEC638, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) Err#10 ECHILD 4394/1: sigaction(SIGCLD, 0xFFBEC5F8, 0xFFBEC678) = 0 4394/1: read(3, " d l\n", 1024) = 3 4394/1: sigaction(SIGCLD, 0xFFBEC5F8, 0xFFBEC678) = 0 4394/1: waitid(P_ALL, 0, 0xFFBEC638, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) Err#10 ECHILD 4394/1: sigaction(SIGCLD, 0xFFBEC5F8, 0xFFBEC678) = 0 4394/1: read(3, 0xFFBECE80, 1024) = 0 4394/1: ioctl(0, TCGETS, 0x0004F13C) = 0 4394/1: sigaction(SIGCLD, 0xFFBECBC8, 0xFFBECC48) = 0 4394/1: sigaction(SIGCLD, 0xFFBECBC8, 0xFFBECC48) = 0 4394/1: close(3) = 0 4394/1: stat64("/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib/libdl.la", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib/libdl.so", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib/libdl.so", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib/libdl.a", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/usr/local/kinetic/ssl/lib/libdl.la", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/usr/local/kinetic/ssl/lib/libdl.so", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/usr/local/kinetic/ssl/lib/libdl.so", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/usr/local/kinetic/ssl/lib/libdl.a", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/lib/libdl.la", 0x0004F450) Err#2 ENOENT 4394/1: stat64("/lib/libdl.so", 0x0004F450) = 0 4394/1: write(2, 0x0004F710, 87) = 87 4394/1: / u s r / l o c a l / k i n e t i c / s p l i t / h t t p d - 2 4394/1: . 0 . 5 8 / s r c l i b / a p r / l i b t o o l [ 4 7 6 7 ] : 4394/1: : i s n o t a n i d e n t i f i e r\n 4394/1: unlink("/tmp/sh4394.6") = 0 4394/1: unlink("/tmp/sh4394.5") = 0 4394/1: unlink("/tmp/sh4394.4") = 0 4394/1: unlink("/tmp/sh4394.3") = 0 4394/1: unlink("") Err#14 EFAULT 4394/1: Incurred fault #6, FLTBOUNDS %pc = 0xFF1C6688 4394/1: siginfo: SIGSEGV SEGV_MAPERR addr=0x00000064 4394/1: Received signal #11, SIGSEGV [default] 4394/1: siginfo: SIGSEGV SEGV_MAPERR addr=0x00000064 4392/1: Received signal #18, SIGCLD, in wait() [caught] 4392/1: siginfo: SIGCLD CLD_DUMPED pid=4394 status=0x000B 4392/1: wait() Err#4 EINTR 4392/1: setcontext(0xFFBEE448) 4392/1: wait() = 4394 [0x008B] 4392/1: write(2, " m a k e", 4) = 4 4392/1: write(2, " [", 1) = 1 4392/1: write(2, " 1 ] : ", 4) = 4 4392/1: write(2, " * * * [", 5) = 5 4392/1: write(2, " h t t p d", 5) = 5 4392/1: write(2, " ] ", 2) = 2 4392/1: write(2, 0xFF2A1DCA, 18) = 18 4392/1: S e g m e n t a t i o n F a u l t 4392/1: write(2, 0x0006C4E4, 14) = 14 4392/1: ( c o r e d u m p e d ) 4392/1: write(2, "\n", 1) = 1 4392/1: stat64("httpd", 0xFFBEE600) Err#2 ENOENT 4392/1: sigprocmask(SIG_BLOCK, 0x00073460, 0x00000000) = 0 4392/1: sigprocmask(SIG_SETMASK, 0xFFBEE6F0, 0x00000000) = 0 4392/1: chdir("/usr/local/kinetic/split/httpd-2.0.58") = 0 4392/1: write(1, 0x00076F9C, 67) = 67 4392/1: m a k e [ 1 ] : L e a v i n g d i r e c t o r y ` / u s r 4392/1: / l o c a l / k i n e t i c / s p l i t / h t t p d - 2 . 0 . 5 4392/1: 8 '\n 4392/1: llseek(0, 0, SEEK_CUR) = 273171 4392/1: _exit(2) 3884/1: waitid(P_ALL, 0, 0xFFBEEB58, WEXITED|WTRAPPED|WSTOPPED) = 0 3884/1: sigaction(SIGCLD, 0xFFBEEB18, 0xFFBEEB98) = 0 3884/1: _exit(1) 3883/1: Received signal #18, SIGCLD, in wait() [caught] 3883/1: siginfo: SIGCLD CLD_EXITED pid=3884 status=0x0001 3883/1: wait() Err#4 EINTR 3883/1: setcontext(0xFFBEE4E8) 3883/1: wait() = 3884 [0x0100] 3883/1: write(2, " m a k e", 4) = 4 3883/1: write(2, " : ", 2) = 2 3883/1: write(2, " * * * [", 5) = 5 3883/1: write(2, 0x000872B8, 13) = 13 3883/1: a l l - r e c u r s i v e 3883/1: write(2, " ] E r r o r ", 8) = 8 3883/1: write(2, " 1", 1) = 1 3883/1: write(2, "\n", 1) = 1 3883/1: sigprocmask(SIG_BLOCK, 0x00073460, 0x00000000) = 0 3883/1: sigprocmask(SIG_SETMASK, 0xFFBEE790, 0x00000000) = 0 3883/1: chdir("/usr/local/kinetic/split/httpd-2.0.58") = 0 3883/1: llseek(0, 0, SEEK_CUR) = 273205 3883/1: _exit(2) 4394/1: *** process killed ***
If you execute the libtool command that fails with a segfault by hand as follows it works: ksh -vx /usr/local/kinetic/split/httpd-2.0.58/srclib/apr/libtool --silent --mode=link gcc -g -O2 -pthreads -DSOLARIS2=7 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -DAP_HAVE_DESIGNATED_INITIALIZER -I/usr/local/kinetic/split/httpd-2.0.58/srclib/apr/include -I/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/include -I/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib -I. -I/usr/local/kinetic/split/httpd-2.0.58/os/unix -I/usr/local/kinetic/split/httpd-2.0.58/server/mpm/worker -I/usr/local/kinetic/split/httpd-2.0.58/modules/http -I/usr/local/kinetic/split/httpd-2.0.58/modules/filters -I/usr/local/kinetic/split/httpd-2.0.58/modules/proxy -I/usr/local/kinetic/split/httpd-2.0.58/include -I/usr/local/kinetic/split/httpd-2.0.58/modules/generators -I/usr/local/kinetic/ssl/include/openssl -I/usr/local/kinetic/ssl/include -I/usr/local/kinetic/split/httpd-2.0.58/modules/dav/main -export-dynamic -L/usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib -L/usr/local/kinetic/ssl/lib -R/usr/local/kinetic/ssl/lib -o httpd modules.lo modules/aaa/mod_access.la modules/aaa/mod_auth.la modules/filters/mod_include.la modules/loggers/mod_log_config.la modules/metadata/mod_env.la modules/metadata/mod_usertrack.la modules/metadata/mod_setenvif.la modules/http/mod_http.la modules/http/mod_mime.la modules/generators/mod_status.la modules/generators/mod_autoindex.la modules/generators/mod_asis.la modules/generators/mod_cgid.la modules/generators/mod_cgi.la modules/mappers/mod_negotiation.la modules/mappers/mod_dir.la modules/mappers/mod_imap.la modules/mappers/mod_actions.la modules/mappers/mod_userdir.la modules/mappers/mod_alias.la modules/mappers/mod_so.la server/mpm/worker/libworker.la server/libmain.la os/unix/libos.la -lssl -lcrypto /usr/local/kinetic/split/httpd-2.0.58/srclib/pcre/libpcre.la /usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/libaprutil-0.la /usr/local/kinetic/split/httpd-2.0.58/srclib/apr-util/xml/expat/lib/libexpat.la -liconv /usr/local/kinetic/split/httpd-2.0.58/srclib/apr/libapr-0.la -lrt -lm -lsocket -lnsl -lresolv -lpthread -ldl
I'd guess the choice of shell is the issue here; this can change between versions of configure/libtool. If it works with ksh, does it also work with /bin/sh? What #! line is the libtool script using? What is the SHELL= line in the libtool script?
The generated libtool shell script is using ksh and works fine for all other uses in the build except for this one instance. The libtool script core dumps when run within the build, when the libtool cmd line above is executed, even when ksh is used to invoke the libtool script. The only way I have been able to get libtool to work for this single case is by executing it by hand with "ksh -vx". Here is the #! line from srclib/apr/libtool: #! /bin/ksh Here is the SHELL= line: # Shell to use when invoking shell scripts. SHELL="/bin/ksh"
I get the same issue - Solaris 7, httpd-2.0.58, same error. This is a Sun4u.
David, do you also get the error before the segfault? /usr/local/kinetic/split/httpd-2.0.58/srclib/apr/libtool[4767]: : is not an identifier Can someone attach the generated srclib/apr/libtool script to see what the line 4767 is?
Adding David to CC, please see previous comment.
Does the libtool command work or fail if you use "ksh -x" only, and not -v? If yes, please post all output, and attach that exact libtool script. Thanks, Ralf
This same problem happened to me on an old Solaris 7 server when building a newer version of mysql. So this build problem isn't limited just to apache2 but must be in some build tool resources both share.
Created attachment 18346 [details] Offending libtool
Ralph: I don't know, I used /bin/sh instead. Joe: Yes, same error. /opt/httpd-2.0.58/srclib/apr/libtool --silent --mode=link gcc -g -O2 -pthreads -DSOLARIS2=7 -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -DAP_HAVE_DESIGNATED_INITIALIZER -I/opt/httpd-2.0.58/srclib/apr/include -I/opt/httpd-2.0.58/srclib/apr-util/include -I/usr/local/include -I. -I/opt/httpd-2.0.58/os/unix -I/opt/httpd-2.0.58/server/mpm/prefork -I/opt/httpd-2.0.58/modules/http -I/opt/httpd-2.0.58/modules/filters -I/opt/httpd-2.0.58/modules/proxy -I/opt/httpd-2.0.58/include -I/opt/httpd-2.0.58/modules/generators -I/usr/local/ssl/include/openssl -I/usr/local/ssl/include -I/opt/httpd-2.0.58/modules/dav/main -export-dynamic -L/usr/local/lib -R/usr/local/lib -L/usr/local/ssl/lib -R/usr/local/ssl/lib -o httpd modules.lo modules/aaa/mod_access.la modules/aaa/mod_auth.la modules/filters/mod_deflate.la modules/loggers/mod_log_config.la modules/metadata/mod_env.la modules/metadata/mod_expires.la modules/metadata/mod_headers.la modules/metadata/mod_setenvif.la modules/http/mod_http.la modules/http/mod_mime.la modules/generators/mod_status.la modules/generators/mod_autoindex.la modules/generators/mod_asis.la modules/generators/mod_suexec.la modules/generators/mod_cgi.la modules/mappers/mod_negotiation.la modules/mappers/mod_dir.la modules/mappers/mod_imap.la modules/mappers/mod_actions.la modules/mappers/mod_alias.la modules/mappers/mod_so.la server/mpm/prefork/libprefork.la server/libmain.la os/unix/libos.la -lz -lssl -lcrypto /opt/httpd-2.0.58/srclib/pcre/libpcre.la /opt/httpd-2.0.58/srclib/apr-util/libaprutil-0.la -lexpat -liconv /opt/httpd-2.0.58/srclib/apr/libapr-0.la -lrt -lm -lsocket -lnsl -lresolv -lpthread -ldl /opt/httpd-2.0.58/srclib/apr/libtool[4767]: : is not an identifier *** Signal 11 - core dumped make: Fatal error: Command failed for target `httpd' Current working directory /opt/httpd-2.0.58 *** Error code 1 make: Fatal error: Command failed for target `all-recursive'
Confirmed. I can reproduce this.
Created attachment 18347 [details] some random patch to make the segfault vanish Well, this shell messes up its memory somewhere. For me, the attached patch makes the segfault go away, but don't ask about a reason (and put away the coffee before looking at the patch). I'd be curious if it helps you as well (probably not, as I guess it's just hiding the bug).
Thanks Ralf. Is that patch going in to 1.5.x upstream?
(In reply to comment #13) > Is that patch going in to 1.5.x upstream? Probably not, as it's not a proper fix but just unexposing the segfault; any other random change before 1.5.24 may re-expose it, so there is little point until we know more about what triggers the memory corruption (or are immediately releasing, so that such hacks are somewhat justifiable). I've been seeing similar failures in different circumstances, and collected a list of these mysterious issues, which are hopefully all due to the same bug; see this thread: http://lists.gnu.org/archive/html/autoconf/2006-05/msg00141.html and there is more, possibly helpful information in this thread for at least one of the item in the list: http://lists.gnu.org/archive/html/bug-autoconf/2006-06/msg00013.html I haven't had time to follow all the clues yet, nor look at ksh source. I suppose one proper workaround would be to use bash for configuring apr libtool when bash is available. This can be achieved by CONFIG_SHELL=/bin/bash # or whereever it is export CONFIG_SHELL $CONFIG_SHELL ./configure CONFIG_SHELL=$CONFIG_SHELL (Yes, the redundancy here is intended; kind of a workaround for a current Autoconf limitation plus future proofness.) Cheers, Ralf
Given that this is a bug in the native ksh and if a workaround were even feasible, would be needed in libtool, closing this as WONTFIX (for want of a CANTFIX).