Bug 45510

Summary: Apache2.2.9 with mod_proxy: Connection reset by peer
Product: Apache httpd-2 Reporter: Mao Morimoto <yneko>
Component: mod_proxyAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: RESOLVED FIXED    
Severity: normal    
Priority: P2    
Version: 2.2.9   
Target Milestone: ---   
Hardware: PC   
OS: Linux   

Description Mao Morimoto 2008-07-30 21:37:01 UTC
I have a backend server(windows) and a reverse proxy(apache 2.2.9). It works almost well, but sometimes cause "Proxy error".

I found some error log that may be relating to this problem.

[Thu Jul 31 11:58:45 2008] [error] [client xxx.xxx.xxx.xxx] (104)Connection reset by peer: proxy: error reading status line from remote server 192.168.1.100, referer: http://example.com/apps/
[Thu Jul 31 11:58:45 2008] [error] [client xxx.xxx.xxx.xxx] proxy: Error reading from remote server returned by /example.gif, referer: http://example.com/apps/

and the config is here;

ProxyRequests Off
<Location /apps/>
    ProxyPass               http://192.168.1.100/
    ProxyPassReverse        http://192.168.1.100/
</Location>

I have investigated the cause, and I think this problem happens right after the backend server closes connections by keep-alive timeout.

Is there any way to solve this problem?
Comment 1 Mao Morimoto 2008-07-30 22:29:46 UTC
It seems the problem happens more frequently when using with SSL.
i.e.
  Backend server(HTTP) <==> (TTTP)Proxy(HTTPS) <==> (HTTPS)Browser
Comment 2 Ruediger Pluem 2008-07-30 22:58:05 UTC
Please set the loglevel to debug and share the results.
Comment 3 Mao Morimoto 2008-07-31 00:13:36 UTC
(In reply to comment #2)

OK, Here is the debug log;

Here is the debug log;

## the previous access (success)
[Thu Jul 31 15:48:31 2008] [debug] proxy_util.c(2102): proxy: connecting http://192.168.1.100/example.gif to 192.168.1.100:80
[Thu Jul 31 15:48:31 2008] [debug] proxy_util.c(2195): proxy: connected /example.gif to 192.168.1.100:80
[Thu Jul 31 15:48:31 2008] [debug] mod_proxy_http.c(1779): proxy: header only
[Thu Jul 31 15:48:31 2008] [debug] proxy_util.c(2062): proxy: HTTP: has released connection for (192.168.1.100)
[Thu Jul 31 15:48:31 2008] [info] [client xxx.xxx.xxx.xxx] Connection closed to child 3 with unclean shutdown (server example.com:443)

... wait 3 minutes to the backend server gets keep-alive timeouts ...

## now all connections to the backend has been closed by peer.
## try to access again
[Thu Jul 31 15:51:29 2008] [info] [client xxx.xxx.xxx.xxx] Connection to child 4 established (server example.com:443)
[Thu Jul 31 15:51:29 2008] [info] Seeding PRNG with 144 bytes of entropy
[Thu Jul 31 15:51:29 2008] [debug] ssl_engine_kernel.c(1752): OpenSSL: Handshake: start

... long long SSL related log ...

[Thu Jul 31 15:51:29 2008] [info] Initial (No.1) HTTPS request received for child 4 (server example.com:443)
[Thu Jul 31 15:51:29 2008] [debug] mod_proxy_http.c(56): proxy: HTTP: canonicalising URL //192.168.1.100/example.gif
[Thu Jul 31 15:51:29 2008] [debug] proxy_util.c(1488): [client xxx.xxx.xxx.xxx] proxy: http: found worker http://192.168.1.100/ for http://192.168.1.100/example.gif
[Thu Jul 31 15:51:29 2008] [debug] mod_proxy.c(966): Running scheme http handler (attempt 0)
[Thu Jul 31 15:51:29 2008] [debug] mod_proxy_http.c(1909): proxy: HTTP: serving URL http://192.168.1.100/example.gif
[Thu Jul 31 15:51:29 2008] [debug] proxy_util.c(2044): proxy: HTTP: has acquired connection for (192.168.1.100)
[Thu Jul 31 15:51:29 2008] [debug] proxy_util.c(2102): proxy: connecting http://192.168.1.100/example.gif to 192.168.1.100:80
[Thu Jul 31 15:51:29 2008] [debug] proxy_util.c(2195): proxy: connected /example.gif to 192.168.1.100:80
[Thu Jul 31 15:51:29 2008] [error] [client xxx.xxx.xxx.xxx] (104)Connection reset by peer: proxy: error reading status line from remote server 192.168.1.100
[Thu Jul 31 15:51:29 2008] [debug] mod_proxy_http.c(1415): [client xxx.xxx.xxx.xxx] proxy: NOT Closing connection to client although reading from backend server 192.168.1.100 failed.
[Thu Jul 31 15:51:29 2008] [error] [client xxx.xxx.xxx.xxx] proxy: Error reading from remote server returned by /apps/example.gif
[Thu Jul 31 15:51:29 2008] [debug] proxy_util.c(2062): proxy: HTTP: has released connection for (192.168.1.100)
[Thu Jul 31 15:51:29 2008] [info] [client xxx.xxx.xxx.xxx] Connection closed to child 4 with unclean shutdown (server example.com:443)
Comment 4 Ruediger Pluem 2008-07-31 03:35:30 UTC
What is your backend server? An IIS?
Comment 5 Mao Morimoto 2008-07-31 04:52:26 UTC
(In reply to comment #4)
> What is your backend server? An IIS?

Yes, an IIS, maybe Windows 2003 Server.
The backend works normally. No error occurs when accessing directly.
Comment 6 Ruediger Pluem 2008-07-31 13:14:59 UTC
*Sigh*. This looks like a resurrection of the famous PR3770. Thus I will mark this bug as a duplicate of PR 37770. We should continue our discussion there. Can you please setup a network sniffer and sniff port 80 and IP 192.168.1.100 between your reverse proxy and you backend while reproducing the bug?

*** This bug has been marked as a duplicate of bug 37770 ***
Comment 7 Teerapat Phetchoochart 2016-09-13 10:03:15 UTC
From this issue . I met other root cause on network config is MTU configuration.this hope for help you all