If Apache server is killed by signal, it leaves shared memory segment (for me with ID 0x0103295c) which is not cleaned up. Also, if Apache is then restarted, it gives the following error: [Mon Apr 08 17:07:03 2002] [crit] (17)File exists: unable to create scoreboard (name-based shared memory failure) and dies with segmentation fault with the following trace: #0 0x4018d109 in chunk_free (ar_ptr=0x40221cc0, p=0x80f0c78) at malloc.c:3111 #1 0x4018cf9a in __libc_free (mem=0x80f0c80) at malloc.c:3023 #2 0x4003dd8a in apr_allocator_destroy (allocator=0x80a9430) at apr_pools.c:135 #3 0x4003e391 in apr_pool_destroy (pool=0x80a94a8) at apr_pools.c:687 #4 0x4003deeb in apr_pool_terminate () at apr_pools.c:513 #5 0x4003a3f5 in apr_terminate () at start.c:112 #6 0x807cb22 in destroy_and_exit_process (process=0x80ad538, process_exit_value=0) at main.c:245 #7 0x807d42c in main (argc=4, argv=0xbffff6a4) at main.c:629 #8 0x4014b9cb in __libc_start_main (main=0x807ce48 <main>, argc=4, argv=0xbffff6a4, init=0x805eaa0 <_init>, fini=0x80920fc <_fini>, rtld_fini=0x4000aea0 <_dl_fini>, stack_end=0xbffff69c) at ../sysdeps/generic/libc-start.c:92 Note also: #1 0x4018cf9a in __libc_free (mem=0x80f0c80) at malloc.c:3023 3023 in malloc.c (gdb) #2 0x4003dd8a in apr_allocator_destroy (allocator=0x80a9430) at apr_pools.c:135 135 free(node); (gdb) p node $1 = (apr_memnode_t *) 0x0 I.e., it does free to NULL. The source is httpd-2.0.35 release, with prefork MPM.
I had the same problem, truss is: from: scoreboard.c, create_namebased_scoreboard: 9935: unlink("/usr/local/apache2/logs/apache_runtime_status") Err#2 ENOENT from: create_namebased_scoreboard, else path of filename == NULL, #if APR_USE_SHMEM_SHMGET 9935: open("/usr/local/apache2/logs/apache_runtime_status", O_WRONLY|O_CREAT|O_EXCL, 0666) = 29 9935: stat64("/usr/local/apache2/logs/apache_runtime_status", 0xFFBEF8A0) = 0 9935: shmget(16778794, 196888, 0600|IPC_CREAT|IPC_EXCL) Err#17 EEXIST The following change helped: remove the IPC_EXCL in: if ((new_m->shmid = shmget(shmkey, new_m->realsize, SHM_R | SHM_W | IPC_CREAT | IPC_EXCL )) < 0) { The scoreboard file was just created with APR_CREATE | APR_EXCL so I think this should be safe. A clean solution would be a function in shm.c which does a cleanup if only the filename is know and use this instead of the apr_file_remove in scoreboard.c erich.prchal@siemens.at
I guess the solution may be to add shmctl(new_m->shmid, IPC_RMID, NULL) just after shmat, like all other shmem-using tools do. Thus pre-removing shared memory segment and making it be released as soon as all processes detach from it. Am I missing something important here?
I have also enountered this bug on Solaris 8/SPARC. A workaround on that platform which allows Apache to be restarted without rebooting is to use /usr/bin/ipcs to find the ID of the shared memory and /usr/bin/ipcrm to remove it. Accordingly, I have changed the Platform/OS of this bug from PC/Linux to All/All and upped the Severity to Major as this is the sort of bug which might cause less experienced sysadmins to reboot when it's not really necessary.
This is one of the main drawbacks of using System V SHM. Try using AcceptMutex pthread if your platform supports it (Solaris certainly does). You may also try "flock" or "fcntl" if your OS supports those values. We can attempt to try to clean up certain shutdown paths, but I'm not sure we can catch everything. The reason we do not want to delete the segment on initialization is that doing so prevents third-party modules (such as mod_perl which relies on this, AIUI) from attaching to the scoreboard externally. Also fix typo in bug summary.
[adding myself to CC]
I am running the following procedure before starting the web server, but the web server doesn't always start... It still gives an error. What am I doing wrong? Linux 2.4.18 [Thu Apr 18 16:03:40 2002] [crit] (17)File exists: unable to create scoreboard (name-based shared memory failure) sub clean_shared_memory { if (! open (PROC, "<", "/proc/sysvipc/shm")) { warn "Cannot open proc filesystem: $!\n"; return; } # key shmid perms size cpid lpid nattch uid gid cuid cgid atime dtime ctime while (<PROC>) { my ($shmid, $uid) = (split (/\s+/))[2,8]; next if ($uid ne "405"); if (! shmctl($shmid, IPC_RMID, 0)) { warn "Cannot remove shared memory segment=<$shmid>: $!\n"; return; } print "Removed shared memory segment=<$shmid>\n"; } close (PROC); }
A few very good proposals have been made here, but unfortunately they are insufficient: 1) calling shmctl(new_m->shmid, IPC_RMID, NULL) right after calling shmat() -- This won't work because segments that are marked IPC_RMID can not be attached to in child processes. 2) calling shmctl($shmid, IPC_RMID, 0) before creating a new segment -- This is also problematic because this would allow a second instance of apache to remove the segment of another already-running apache. It is my understanding that the normal shutdown signals will call the cleanup routines and properly detach and remove the segment. If you are killing apache with SIGKILL then you're going to have to use ipcs/ipcrm to clean up those segments. For the above reasons I'm marking this bug as invalid, if you disagree please comment.
What is the proper way to kill apache, having it NOT finish servicing any requests, but die immediately and exit cleanly? This is what I do now, which apparently leaves shared memory around sometimes. if (-e /var/run/httpd.pid) { kill -TERM `cat /var/run/httpd.pid` rm /var/run/httpd.pid } sleep 2 killall -9 httpd
The proper way is to use httpd -k stop or apachectl -k stop. This should run all the cleanup stuff and shutdown gently.