Bug 52759 - Apache httpd child process consuming high CPU Utilization.
Summary: Apache httpd child process consuming high CPU Utilization.
Status: RESOLVED WORKSFORME
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: All (show other bugs)
Version: 2.2.11
Hardware: Other Linux
: P2 critical (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-02-24 10:44 UTC by Hiren Patel
Modified: 2016-05-16 15:15 UTC (History)
0 users


Attachments
output of lsof -p <pid> (6.72 KB, text/plain)
2012-02-24 10:44 UTC, Hiren Patel 		Details
pstack and gdb trace output... (7.89 KB, text/plain)
2012-03-09 10:35 UTC, Hiren Patel 		Details
updated backtrace output of gdb (20.74 KB, text/plain)
2012-03-09 10:45 UTC, Hiren Patel 		Details
ps output (100.51 KB, image/pjpeg)
2012-03-09 10:50 UTC, Hiren Patel 		Details
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Hiren Patel 2012-02-24 10:44:44 UTC 
Created attachment 28378 [details]
output of lsof -p <pid>

Hello Team,

We are facing high cpu utilization issue for Apache httpd process(child)

Apache version:2.2.11

Tomcat version:6.0.18

OS:linux 

uname-a :Linux WEB10_64 2.6.18-92.el5 #1 SMP Tue Apr 29 13:16:15 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux

lsof output is attached below.

-->Apache childprocess is waiting for closed state hence it is consuming high CPU.
Comment 1 Eric Covener 2012-02-24 12:20:31 UTC 
please collect and post periodic backtraces and ps output.
Comment 2 Hiren Patel 2012-03-09 10:35:49 UTC 
Created attachment 28444 [details]
pstack and gdb trace output...
Comment 3 Hiren Patel 2012-03-09 10:45:41 UTC 
Created attachment 28445 [details]
updated backtrace output of gdb
Comment 4 Hiren Patel 2012-03-09 10:50:08 UTC 
Created attachment 28446 [details]
ps output

output of ps command...
Comment 5 Rainer Jung 2012-03-09 18:57:11 UTC 
The problem seems to happen, when mod_jk reveiced the final package from the AJP back end and at the end of the response request cycle calls ap_finalize_request_protocol(). The connection to the client is an HTTPS connection and maybe it happens during lingering close? Was the lsof taken during the spining CPU? The lsof output does not contain the client connection, so it seems that connection was already closed.

The loop seems to be in OpenSSL. What looks suspicious in the dumps is len=0x7fff95282a50 in ssl_io_input_read().

I don't have a real recommendation here.

Rainer
Comment 6 Hiren Patel 2012-03-10 05:46:02 UTC 
Yes, lsof output was taken when Apache process was taking high cpu..
openssl version is 0.9.8b

Is this issue related to socket connection or timeouts as in worker.properties we have not set any timeouts??

or is it related to application which is not closing connections properly??

I have gone through many forums for this issue...

setting value for CFLAGS=NO_LINGERCLOSE and then building apache again..

whether increasing the ulimit (no of open files) help ??

Please help as this issue is impacting production servers...
Comment 7 Rainer Jung 2012-03-10 10:55:03 UTC 
I don't see any relation to mod_jk settings. The stacks show the code is looping in the HTTPS handling of the client/browser connection. The lsof output shows, that the connection no longer exists. So it seems some error condition is not handled properly around OpenSSL/mod_ssl.

Your Apache and OpenSSL versions are pretty old. Any chance to update and retest? It might well be some problem fixed long ago.

Regards,

Rainer
Comment 8 Eric Covener 2016-05-16 15:15:39 UTC 
No news in 4 years...