I didn't see "mod_mem_cache" in the Component field so assigning this to mod_cache. I have an Apache 2.2.2 setup on a Redhat box. Mod_proxy(mod_proxy_ajp) is being used to connect to tomcat on the same machine. Caching is implemented using mod_cache(mod_mem_cache). This setup works fine and caching seems to be working as expected. The problem occurs when I try to optimize further by supporting HTML compression using mod_deflate. In this case, whenever deflated documents are served by the cache the Content-length returned is 0. Is this a known issue ? Can mod_deflate and mod_mem_cache be used together ? Or is it that mod_mem_cache cannot handle compressed content ? Here's the output from lwp-request( the second one shows the serving up of cached content) [root@bin]# lwp-request -uedsx http://localhost:80/Main.do -H Accept-Encoding:gzip,deflate LWP::UserAgent::new: () LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://localhost:80/Main.do LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 670 bytes LWP::Protocol::collect: read 4096 bytes LWP::Protocol::collect: read 3002 bytes LWP::UserAgent::request: Simple response: OK GET http://localhost:80/Main.do 200 OK Cache-Control: no-store, must-revalidate, post-check=0, pre-check=0 Connection: close Date: Tue, 18 Jul 2006 18:27:35 GMT Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Content-Length: 7768 Content-Type: text/html;charset=ISO-8859-1 Expires: Tue, 18 Jul 2006 18:30:37 GMT Last-Modified: Tue, 18 Jul 2006 18:27:37 GMT Client-Date: Tue, 18 Jul 2006 18:27:37 GMT Client-Peer: 127.0.0.1:80 Client-Response-Num: 1 [root@bin]# lwp-request -uedsx http://localhost:80/Main.do -H Accept-Encoding:gzip,deflate LWP::UserAgent::new: () LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://localhost:80/Main.do LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::UserAgent::request: Simple response: OK GET http://localhost:80/Main.do 200 OK Cache-Control: no-store, must-revalidate, post-check=0, pre-check=0 Connection: close Date: Tue, 18 Jul 2006 18:28:32 GMT Age: 56 Server: Apache/2.2.2 (Unix) Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Content-Length: 0 Content-Type: text/html;charset=ISO-8859-1 Expires: Tue, 18 Jul 2006 18:30:37 GMT Last-Modified: Tue, 18 Jul 2006 18:27:37 GMT Client-Date: Tue, 18 Jul 2006 18:28:32 GMT Client-Peer: 127.0.0.1:80 Client-Response-Num: 1 Here's the deflate.log contents "GET /Main.do HTTP/1.1" 7750/35169 (22%) "GET /Main.do HTTP/1.1" -/- (-%) Relevant sections of my httpd.conf: LoadModule cache_module modules/mod_cache.so LoadModule mem_cache_module modules/mod_mem_cache.so LoadModule deflate_module modules/mod_deflate.so LoadModule expires_module modules/mod_expires.so LoadModule headers_module modules/mod_headers.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_ajp_module modules/mod_proxy_ajp.so # # mod_expires settings # <IfModule expires_module> ExpiresActive On ExpiresByType text/css "access plus 1 day" ExpiresByType image/gif "access plus 1 month" ExpiresByType image/jpeg "access plus 1 month" </IfModule> # # mod_proxy/mod_proxy ajp settings # <IfModule proxy_module> ProxyRequests Off <Proxy *> Order deny,allow Allow from all </Proxy> ProxyPass /favicon.ico ! ProxyPass / ajp://localhost:8009/ ProxyPassReverse / ajp://localhost:8009/ </IfModule> # # mod_cache/mod_mem_cache settings # <IfModule cache_module> <IfModule mem_cache_module> CacheEnable mem / CacheStoreNoStore On # CacheIgnoreCacheControl On # CacheIgnoreNoLastMod On # CacheMaxExpire 150 MCacheSize 4096 MCacheMaxObjectCount 200 MCacheMinObjectSize 1 MCacheMaxObjectSize 524288 </IfModule> </IfModule> # # Worker MPM settings # <IfModule mpm_worker_module> StartServers 1 MaxClients 250 ThreadsPerChild 50 MinSpareThreads 25 MaxSpareThreads 75 </IfModule> <IfModule deflate_module> SetOutputFilter DEFLATE SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png|rar|zip)$ no-gzip DeflateFilterNote Input instream DeflateFilterNote Output outstream DeflateFilterNote Ratio ratio LogFormat '"%r" %{outstream}n/%{instream}n (%{ratio}n%%)' deflate CustomLog logs/deflate.log deflate BrowserMatch ^Mozilla/4 gzip-only-text/html BrowserMatch ^Mozilla/4\.0[678] no-gzip BrowserMatch \bMSIE !no-gzip !gzip-only-text/html <IfModule headers_module> Header append Vary User-Agent </IfModule> </IfModule> I posted this issue in the user support list. As suggested there, I configured apache to use mod_disk_cache and it seems to be working and compressed content is getting served up correctly. So I am guessing that this means it is a bug in mod_mem_cache. Please let me know if you need more information.
If you want this to work, try using mod_disk_cache, which is known to be working correctly with mod_deflate.
This also happens with mod_disk_cache. The reason is that the first filter in the filter chain that we kick off in the quick handler for a cached entity is the deflate filter. This filter adds an eos bucket to our empty brigade. This results in a brigade that looks like the following being delivered to the content_length filter: EOS bucket File bucket EOS bucket Thus the content_length filter calculates 0 as Content-Length. While one might argue that the behaviour of the deflate output filter is broken and that it should not add an eos bucket to the brigade if it has not seen one, I think we should start the filter chain always with our cache out filter when we kick off the filter chain in the quick handler, as this is the point where we actually saved the cached data during caching. Please give the attached patch a try (patch against trunk, but also works against 2.2.2) and let me know if it solves your problem.
Created attachment 18627 [details] Patch against trunk
(In reply to comment #2) > This also happens with mod_disk_cache. The reason is that the first filter in > the filter chain that we kick off in the quick handler for a cached entity is > the deflate filter. This filter adds an eos bucket to our empty brigade. This I must correct myself. The deflate out filter returns immediately without passing the empty brigade down the chain. So the EOS at the start of the brigade that we are seeing is actually caused by ap_finalize_request_protocol. But my further conclusions are still valid.
Ruediger Pluem, Thank you very much for the fix. Sorry I couldn't test your patch before today. Anyways, I applied the patch and ran some tests. It seems to have fixed the issue that was reported. But, I noticed another problem. Here's what's happening... ############# First Request ( Works fine-Caches the compressed content correctly ) ############# [root@local]# lwp-request -uedsx http://localhost:80/Main.do -H Accept-Encoding:gzip,deflate LWP::UserAgent::new: () LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://localhost:80/Main.do LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 670 bytes LWP::Protocol::collect: read 4096 bytes LWP::Protocol::collect: read 2836 bytes LWP::UserAgent::request: Simple response: OK GET http://localhost:80/Main.do 200 OK Cache-Control: no-store, must-revalidate, post-check=0, pre-check=0 Connection: close Date: Tue, 25 Jul 2006 18:42:12 GMT Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Content-Length: 7602 Content-Type: text/html;charset=ISO-8859-1 Expires: Tue, 25 Jul 2006 18:45:13 GMT Last-Modified: Tue, 25 Jul 2006 18:42:13 GMT Client-Date: Tue, 25 Jul 2006 18:42:13 GMT Client-Peer: 127.0.0.1:80 Client-Response-Num: 1 error_log ######### [Tue Jul 25 11:42:12 2006] [debug] mod_cache.c(131): Adding CACHE_SAVE filter for /Main.do [Tue Jul 25 11:42:12 2006] [debug] mod_cache.c(138): Adding CACHE_REMOVE_URL filter for /Main.do ***snip fetches data correctly*** [Tue Jul 25 11:42:13 2006] [debug] mod_deflate.c(447): [client 127.0.0.1] Zlib: Compressed 35547 to 7584 : URL /Main.do [Tue Jul 25 11:42:13 2006] [debug] mod_headers.c(612): headers: ap_headers_output_filter() [Tue Jul 25 11:42:13 2006] [debug] mod_cache.c(621): cache: Caching url: /Main.do [Tue Jul 25 11:42:13 2006] [debug] mod_cache.c(627): cache: Removing CACHE_REMOVE_URL filter. [Tue Jul 25 11:42:13 2006] [debug] mod_cache.c(670): cache: Added date header [Tue Jul 25 11:42:13 2006] [info] mem_cache: Cached url: http://localhost:80/Main.do? ############### Second Request ( Works fine-Fetches compressed content from cache ) ############## [root@local]# lwp-request -uedsx http://localhost:80/Main.do -H Accept-Encoding:gzip,deflate LWP::UserAgent::new: () LWP::UserAgent::request: () LWP::UserAgent::send_request: GET http://localhost:80/Main.do LWP::UserAgent::_need_proxy: Not proxied LWP::Protocol::http::request: () LWP::Protocol::collect: read 632 bytes LWP::Protocol::collect: read 4096 bytes LWP::Protocol::collect: read 2874 bytes LWP::UserAgent::request: Simple response: OK GET http://localhost:80/Main.do 200 OK Cache-Control: no-store, must-revalidate, post-check=0, pre-check=0 Connection: close Date: Tue, 25 Jul 2006 18:42:30 GMT Age: 18 Server: Apache/2.2.2 (Unix) Vary: Accept-Encoding,User-Agent Content-Encoding: gzip Content-Length: 7602 Content-Type: text/html;charset=ISO-8859-1 Expires: Tue, 25 Jul 2006 18:45:13 GMT Last-Modified: Tue, 25 Jul 2006 18:42:13 GMT Client-Date: Tue, 25 Jul 2006 18:42:30 GMT Client-Peer: 127.0.0.1:80 Client-Response-Num: 1 error_log ######### [Tue Jul 25 11:42:30 2006] [debug] mod_cache.c(280): cache: running CACHE_OUT filter [Tue Jul 25 11:42:30 2006] [debug] mod_cache.c(294): cache: serving /Main.do ############# Third Request ( Problem-It correctly detects a vary header mismatch but doesn't cache the results. ) ############# I make the third request using a different "User-Agent" ( firefox in this case ) error_log ######### [Tue Jul 25 11:43:18 2006] [debug] cache_storage.c(246): cache_select_url(): Vary header mismatch. [Tue Jul 25 11:43:18 2006] [debug] mod_cache.c(131): Adding CACHE_SAVE filter for /Main.do [Tue Jul 25 11:43:18 2006] [debug] mod_cache.c(138): Adding CACHE_REMOVE_URL filter for /Main.do ***snip fetches data correctly*** [Tue Jul 25 11:43:20 2006] [debug] mod_deflate.c(447): [client 127.0.0.1] Zlib: Compressed 35168 to 7751 : URL /Main.do [Tue Jul 25 11:43:20 2006] [debug] mod_headers.c(612): headers: ap_headers_output_filter() I don't see "cache: Caching url: /Main.do" or "cache: Removing CACHE_REMOVE_URL filter" log messages like in the first request.Looks like it caches compressed content only for the first type of "User-Agent" ?
mod_mem_cache cannot handle varied content correctly and a fix will require bigger changes. Thus I leave this report open. In the meantime you can fall back to mod_disk_cache which can handle varied content properly. Regarding possible performance thoughts: In most cases mod_disk_cache is as fast as mod_mem_cache, in some cases it is even faster.
Committed patch to trunk as r425787 (http://svn.apache.org/viewvc?rev=425787&view=rev).
Thanks for the feedback,Ruediger. Apart from using mod_disk_cache, the other option I can think of is to not have the User-Agent in the Vary header. The only reason I have it there is for the BrowserMatch directives for mod_deflate. Maybe I can get rid of those directives ?
*** Bug 40278 has been marked as a duplicate of this bug. ***
Proposed for backport as r489460 (http://svn.apache.org/viewvc?rev489460=&view=rev).
Backported to 2.2.x as r538868 (http://svn.apache.org/viewvc?rev=538868&view=rev).