Bug 31226 - AddOutputFilterByType deflate not active with mod_proxy
Summary: AddOutputFilterByType deflate not active with mod_proxy
Status: CLOSED FIXED
Alias: None
Product: Apache httpd-2
Classification: Unclassified
Component: Core (show other bugs)
Version: 2.0.50
Hardware: PC Linux
: P3 critical (vote)
Target Milestone: ---
Assignee: Apache HTTPD Bugs Mailing List
URL:
Keywords:
: 14335 41146 (view as bug list)
Depends on:
Blocks:
 
Reported: 2004-09-14 18:56 UTC by frederic bregier
Modified: 2009-05-22 07:59 UTC (History)
4 users (show)



Attachments
patch against 2.0.54 (775 bytes, patch)
2006-12-11 09:59 UTC, Bill McGonigle
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description frederic bregier 2004-09-14 18:56:42 UTC
I use Apache 2.0 to have a reverse proxy web server that
compress with mod_deflate the web pages in direction of the users.
Applications behind (on application "proxified" server) generate :
- text/html document
- application/pdf or others like jpg document 
both using an url such as http://myreverseproxy/application/application.exe
or application.php whatever like this.

The url is always the same for one application, only the context (session)
implies the answear (html or pdf document).
We want to compress document with type mime = text, but not those
with jpg or pdf type mime.

But here is the problem, with proxypass and proxypassreverse :

1) using SetOutputFilter DEFLATE implies that everything is compressed
even if the mime type is a pdf or jpg since the exclusion can be based 
only on url, which is not possible here.
We can exclude files like http://myreverseproxy/application/image.jpg
but not the same jpg create dynamically by 
http://myreverseproxy/application/application.exe

2) using AddOutputFilterByType DEFLATE text/html text/plain
compressed nothing at all, neither html neither pdf or jpg.
And accordingly to the documentation and sources on the web,
this second solution should be the correct one.
Every mime types are defined and ok, the returned documents
are using the correct mime type (text/html, application/pdf, ...).

I found not much information on the web on a reverse proxy with
compression at the same time.
It is quite a big problem for us since we cannot use
the web compression in this conditions.
Has someone have this problem too ?
What can I do to correct this ?

thank you for your help
Comment 1 Nick Kew 2004-09-15 11:23:55 UTC
AddOutputFilterbyType is known to be broken.  Use mod_filter instead:
http://httpd.apache.org/docs-2.1/mod/mod_filter.html
Comment 2 frederic bregier 2004-09-15 12:22:05 UTC
Thank you for your note on the broken AddOutputFilterByType.
I read the manual page of mod_filter.
First, it is only valid with Apache 2.1. 
Is it enough stable for this use (reverse proxy and deflate) for production ?

Second, I try to think what could be the syntax of the filter.

FilterDeclare compressed Content-Type
FilterProvider compressed deflate $text/html
# is it deflate or mod_deflate ?
FilterProvider compressed deflate $text/plain
FilterProtocol compressed "change=yes"
<Location /application>
FilterChain compressed
</Location> 

But I miss a point. What about the proxy definition ?
Is the proxy (proxypass and reverse) applied before this filter,
or should I make another Filter with the final filterchain as
FilterChain proxyfied compressed ?
Therefore, how can I specify the directives for this filter 
(proxypass and reverse) ?
What about special mod_deflate directives (deflatecompressionlevel for 
instance) ?

Also there is the registering of the modules with ap_register_output_filter.
Is is done with all standard modules and with what names ?
(for mod_proxy, mod_deflate ?)

I know now this module is new, so I can understand that the manual is not 
completed.
Thank you again
Comment 3 Nick Kew 2004-09-15 13:22:33 UTC
This is not really a support forum.  Please direct support questions somewhere
more appropriate, such as the apache-users mailinglist or a suitable webservers
newsgroup.

But in brief,
 (1) it works fine with 2.0
 (2) your filter syntax looks fine to me
 (3) It doesn't affect your proxy setup or your mod_deflate directives.  Stop
looking for complexity where there is none!
Comment 4 frederic bregier 2004-09-16 12:11:44 UTC
Thank you for this help.
For everyone having the same problem and looking in this bug,
I place the solution that works for me.
With standard proxy and deflate directives, I placed the following

FilterDeclare compressed Content-Type
FilterProvider compressed deflate $text/html
FilterProvider compressed deflate $text/plain
FilterProvider compressed deflate $text/css
#FilterProtocol compressed deflate "change=yes"
FilterChain compressed

I had to download spcifically the mod_filter.c through
your web page, Nick (http://www.apache.org/~niq/)
since it is not in Apache sources (neither 2.0 or 2.1).
I understand since the first release is of august 2004.

I comment FilterProtocol since it produces an error
in apache ("FilterProtocol: No such filter").
I try with and without "deflate".

Well, at this time, it seems to work as intended.
Since it is a quite new module, I have to test it intensively
since it is for production later on.

Thank you ! Bug closed !

Comment 5 Nick Kew 2005-10-21 10:28:35 UTC
*** Bug 14335 has been marked as a duplicate of this bug. ***
Comment 6 Ruediger Pluem 2005-10-21 11:39:42 UTC
(In reply to comment #1)
> AddOutputFilterbyType is known to be broken.  Use mod_filter instead:

The question to me is: Why is it broken? While having a look a PR14335 I found
out that the reason for this "brokenness" seems to be the following lines from
ap_add_output_filters_by_type in core.c:

    /* We can't do anything with proxy requests, no content-types or if
     * we don't have a filter configured.
     */
    if (r->proxyreq != PROXYREQ_NONE || !r->content_type ||
        !conf->ct_output_filters) {
        return;
    }

Does anybody remember the reason why we cannot do anything on proxied resources?
This change had been made in r94028 about 3,5 years ago by Bill Stoddard.
Comment 7 Ruediger Pluem 2005-10-21 11:46:29 UTC
(In reply to comment #4)

[..cut..]

> I had to download spcifically the mod_filter.c through
> your web page, Nick (http://www.apache.org/~niq/)
> since it is not in Apache sources (neither 2.0 or 2.1).

It is part of httpd 2.1 and was protmoted from its experimental state to be a
regular module a while ago. The configure switch for mod_filter is --enable-filter.

[..cut..]
Comment 8 Joe Orton 2005-10-21 12:35:04 UTC
I looked at that a while back and nobody could work out why the check is there
either.  If the check was removed basic things seem to work fine.

http://mail-archives.apache.org/mod_mbox/httpd-dev/200409.mbox/%3c20040916161122.GA16320@redhat.com%3e

removing it in 2.2/trunk and seeing what breaks seems reasonable.
Comment 9 Ruediger Pluem 2005-10-21 15:52:19 UTC
I just committed Joe's patch to the trunk (r327179):
http://svn.apache.org/viewcvs.cgi/httpd/httpd/trunk/server/core.c?rev=327179&r1=306495&r2=327179
It fixes the problem.
Comment 11 Joe Orton 2006-12-11 07:45:32 UTC
*** Bug 41146 has been marked as a duplicate of this bug. ***
Comment 12 Bill McGonigle 2006-12-11 09:59:53 UTC
Created attachment 19243 [details]
patch against 2.0.54

adding patch here per Joe Orton on bug 41146.

As noted there, this patch has been applied to a production 2.0.54 server
(fedora core 4) which has served about 14 million deflated pages through the
proxy without problems.
Comment 13 otheus 2008-12-18 17:09:48 UTC
Am experiencing the same problem in 2.2.3 (RHEL5, Linux 2.6.18). The rule which hands this off to the internal proxy is:

 RewriteRule     ^/(.*\.php)$    http://WWW.xxxxx.COM:80/$1    [P]

and compression in particular set for CSS:

 AddOutputFilterByType DEFLATE text/css

(I also tried the Filter module with similar results.) 

Proxying is working fine, deflating is working fine... just not together. According to SVN, the CHANGE_LOG says this was fixed as of 2.2.2. Is it possible something still doesn't work correctly? Is it possible that the rewrite module and this module don't play together well??


Comment 14 Ruediger Pluem 2008-12-19 00:47:21 UTC
1. 2.2.3 is a very old version.
2. I rechecked the code and cannot find something suspicious.

So please post your complete configuration.
Comment 15 Dan Poirier 2009-05-22 07:52:31 UTC
It appears the original problem has been fixed.  If it's seen again (on a current version), this can be reopened.