Bug 53420 - Bug with ProxyErrorOverride / httpd-2.4.x
Summary: Bug with ProxyErrorOverride / httpd-2.4.x
Status: RESOLVED FIXED
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: mod_proxy (show other bugs)
Version: 2.4.1
Hardware: PC Linux
: P2 normal with 2 votes (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-15 07:51 UTC by Zisis Lianas
Modified: 2018-05-07 11:43 UTC (History)
5 users (show)



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zisis Lianas 2012-06-15 07:51:03 UTC
When setting ProxyErrorOverride to "On" to get the httpd-ErrorDocument
instead of the backend-errordoc, httpd is waiting "ProxyTimeout" seconds
to respond to the client, even though the response is already read from
backend server.

The request/response is hanging somewhere in
  module/proxy/mod_proxy_http
    -> ap_proxy_http_process_response
       -> ap_discard_request_body(backend->r) (modules/http/http_filters)
before timing out with "ProxyTimeout".


ProxyErrorOverride Directive:
http://httpd.apache.org/docs/current/mod/mod_proxy.html#proxyerroroverride


I can reproduce this error only in 2.4.x.
2.2.x works correctly.
Comment 1 Luka Dolenec 2013-11-18 13:46:20 UTC
HI.

We come across with same behavior with version 2.4.6
Our configuration is:
ProxyErrorOverride On
ProxyPass /test  http://xxxxxx:8080/test
ProxyPassReverse /test  http://xxxxxx:8080/test
ErrorDocument 404 "Page does not exist"
When we call proxy URL /test/nonexistingpage from browser, browser start to load page few minutes, then display "Page does not exist"
Same configuration on version 2.2.3 it works without problem.

Best regards.
Comment 2 Vijesh Shetty 2014-04-29 06:24:50 UTC
I am running Apache 2.4.7 and I find the same problem here too.
Anything other than 1xx, 2xx, 3xx response, proxy module waits for the default timeout before serving the ErrorDocument page if I have ProxyErrorOverride On.

Did anyone get a solution to this problem?

Thanks,
Vijesh
Comment 3 Viktor Stolbin 2014-07-16 14:18:46 UTC
Does anyone have any update on this? Is there any ticket to fix this?
Comment 4 Rainer Jung 2014-08-01 08:31:25 UTC
First steps of analysis for trunk. The problem happens with all MPMs, especially also with prefork and event.

The hang until timeout happens in ap_discard_request_body called from mod_proxy_http.c in line 1640:

1629 	/* PR 41646: get HEAD right with ProxyErrorOverride */
1630 	if (ap_is_HTTP_ERROR(r->status) && dconf->error_override) {
1631 	/* clear r->status for override error, otherwise ErrorDocument
1632 	* thinks that this is a recursive error, and doesn't find the
1633 	* custom error page
1634 	*/
1635 	r->status = HTTP_OK;
1636 	/* Discard body, if one is expected */
1637 	if (!r->header_only && /* not HEAD request */
1638 	(proxy_status != HTTP_NO_CONTENT) && /* not 204 */
1639 	(proxy_status != HTTP_NOT_MODIFIED)) { /* not 304 */
1640 	ap_discard_request_body(backend->r);
1641 	}
1642 	proxy_run_detach_backend(r, backend);
1643 	return proxy_status;
1644 	}

In ap_discard_request_body - which is used here to discard the reponse body from the origin server - I can see two brigades being read. The first has one bucket which contains the original error page coming form the origin server (here 206 bytes), the second brigade only has a EOS bucket. Reading the second brigade is where it blocks until proxy timeout.

For 2.2 the situation is different: the first brigade read has two buckets, the first again containing the origin server response body, the second one the EOS. The brigade with both buckets could be read immediately without waiting for timeout.

Any ideas what's going one? Hints in which direction to proceed?

Regards,

Rainer
Comment 5 Rainer Jung 2014-08-01 21:30:51 UTC
The reason is broken header handling. Headers are read from the origin server and then directly put into r->headers_out. In 2.2 those headers are copied to the request struct (rp) used in ap_discard_request_body. therefore the http infput filter can detect the end of the body using the Content-Length header and sets EOS as soon as all the expected bytes are read. 

In 2.4 and trunk, copying the headers from r->headers_out to backend->r-headers_in happens *after* the error_override handling. Thus the http input filter must read until timeout.

Either we rearrange the original order of code, or we at least copy Content-Length and Transfer-Encoding headers from r->headers_out to backend->r_headers_in before doing ap_discard_request_body. Somethin like

                const char *tmp;
                if (tmp = apr_table_get(r->headers_out, "Content-Length")) {
                    apr_table_set(backend->r->headers_in, "Content-Length", tmp);
                }
                else if (tmp = apr_table_get(r->headers_out, "Transfer-Encoding")) {
                    apr_table_set(backend->r->headers_in, "Transfer-Encoding", tmp);
                }
                else if (te) {
                    apr_table_set(backend->r->headers_in, "Transfer-Encoding", te);
                }
                ap_discard_request_body(backend->r);
                ...

I will check svn history to get an idea, why the code order was changed.
Comment 6 Rainer Jung 2014-08-01 21:40:14 UTC
The bug was introduced by r912063 to fix PR41646.
Comment 7 Rainer Jung 2014-08-02 08:32:57 UTC
Fix committed in r1615289.
Will propose for backport to 2.4 next week if no negative reactions come up in the meantime.
Comment 8 Vijesh Shetty 2014-08-12 13:53:56 UTC
Is the fix backported to 2.4.x?
Comment 9 Christophe JAILLET 2014-08-12 19:19:10 UTC
It has been proposed for backport for 2.4.x in r1615346 but it has not been accepted yet.
Comment 10 Yann Ylavic 2015-01-23 10:22:41 UTC
Backported to 2.4.11 in r1621601.
Comment 11 cristian.httpd 2015-03-19 00:19:51 UTC
Hi, there are some additional issues with ProxyErrorOverride set to on, please see https://bz.apache.org/bugzilla/show_bug.cgi?id=56925 

Anyone else experienced this, looking at this ticket it seems like the turnaround time may be up to 2 years until it will be solved.