How To Save Traffic With Apache2's mod_deflate

Want to support HowtoForge? Become a subscriber!
 
Submitted by falko (Contact Author) (Forums) on Mon, 2006-05-15 13:40. :: Apache

How To Save Traffic With Apache2's mod_deflate

Version 1.0
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited 05/15/2006

In this tutorial I will describe how to install and configure mod_deflate on an Apache2 web server. mod_deflate allows Apache2 to compress files and deliver them to clients (e.g. browsers) that can handle compressed content which most modern browsers do. With mod_deflate, you can compress HTML, text or XML files to approx. 20 - 30% of their original sizes, thus saving you server traffic and making your modem users happier.

Compressing files causes a slightly higher load on the server, but in my experience this is compensated by the fact that the clients' connection times to your server decrease a lot. For example, a modem user that needed seven seconds to download an uncompressed HTML file might now only need two seconds for the same, but compressed file.

By using mod_deflate you don't have to be afraid that you exclude users with older browsers that cannot handle compressed content. The browser negotiates with the server before any file is transferred, and if the browser does not have the capability to handle compressed content, the server delivers the files uncompressed.

mod_deflate has replaced Apache 1.3's mod_gzip in Apache2. If you want to serve compressed files with Apache 1.3, take a look at this tutorial: mod_gzip - serving compressed content by the Apache webserver

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Enable mod_deflate

If you have Apache2 installed, mod_deflate should also already be installed on your system. Now we have to enable it. On Debian, we can do it like this:

a2enmod deflate

Then restart Apache2:

/etc/init.d/apache2 restart

On other distributions you might have to edit Apache2's configuration manually to enable mod_deflate. You might have to add a line like this to the LoadModule section:

LoadModule deflate_module /usr/lib/apache2/modules/mod_deflate.so

Make sure you adjust the path to mod_deflate.so, and restart Apache2 afterwards.

2 Configure mod_deflate

The compression of files can be configured in one of two ways: either explicit exclusion of files by extension or explicit inclusion of files by MIME type. You can enable mod_deflate for your whole Apache2 server, or just for specific virtual sites. Depending on this, either open your Apache2's global server configuration section now or just the vhost configuration section where you want to enable mod_deflate.

2.1 Explicit Inclusion Of Files By MIME Type

If you want to compress HTML, text, and XML files only, add this line to your configuration:

AddOutputFilterByType DEFLATE text/html text/plain text/xml

This is the configuration I'm using because I don't want to compress images or PDF files or already compressed files such as zip files.

2.2 Explicit Exclusion Of Files By Extension

If you want to compress all file types and exclude just a few, you would add something like this to your configuration (instead of the line from section 2.1):

SetOutputFilter DEFLATE
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ \
no-gzip dont-vary
SetEnvIfNoCase Request_URI \
\.(?:exe|t?gz|zip|bz2|sit|rar)$ \
no-gzip dont-vary
SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary

This would compress all files except images (gif, jpg, and png), already compressed files (like zip and tar.gz) and PDF files which makes sense because you do not gain much by compressing these file types.

2.3 Further Configuration Directives

Regardless whether you use the configuration from section 2.1 or 2.2, you should add these lines to your configuration:

BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

These lines are for some older browsers that do not support compression of files other than HTML documents.

The configuration is now finished, and you must now restart Apache2. On Debian, you do it like this:

/etc/init.d/apache2 restart

To learn about further configuration directives, take a look at Apache Module mod_deflate.

3 Testing

To test our compression, we add a few directives to our mod_deflate configuration that log the compression ratio of delivered files. Open your mod_deflate configuration and add the following lines:

DeflateFilterNote Input input_info
DeflateFilterNote Output output_info
DeflateFilterNote Ratio ratio_info
LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate
CustomLog /var/log/apache2/deflate_log deflate

Make sure you replace /var/log/apache2 with your Apache2's log directory. This could be /var/log/httpd, /var/log/httpd2, etc.

Then restart Apache2. On Debian, do it like this:

/etc/init.d/apache2 restart

Now whenever a file is requested this will be logged in /var/log/apache2/deflate_log (or to whatever file you changed it to). A typical log line looks like this:

"GET /info.php HTTP/1.1" 7621/45430 (16%)

You see that the file info.php was requested and delivered. Its original size was 45430 bytes, and it was compressed to 7621 bytes or 16% of its original size! This is a great result, and if your web site mostly consists out of HTML, text, and XML files, mod_deflate will save you a lot of traffic, and for users with a low-bandwidth connection your site will load much faster.

If you don't need the logging after your tests anymore, you can undo the changes from section 3 and restart Apache2.

4 Links


Please do not use the comment function to ask for help! If you need help, please use our forum.
Comments will be published after administrator approval.
Submitted by cbj4074 (registered user) on Fri, 2012-05-18 14:41.
Submitted by Mikhailov Anatoly (not registered) on Tue, 2008-12-16 09:06.
Submitted by Anonymous (not registered) on Sat, 2006-06-10 10:22.

Your guide works like a breeze, thank you very much!
I do have a slight problem though: in de deflate log, requests that are not zipped (like images) also appear there. Can it be changed so only zipped requests are logged there? That would make a nice way to generate a daily report on how much traffic has been saved.

Submitted by Anonymous (not registered) on Sat, 2006-06-10 09:46.

Would be great if it worked, but mod_deflate has never worked for me on Fedora Core. Fedora Core loads the mod_deflate module by default, so supposively all I have to do is enable it for the mime type text/css - but it just won't compress my CSS at all. I have tried this both on Fedora Core 4 and 5, it really frustrates me that it just won't work and I can't even see why not. Can't find any information on the net either.

In the end I ended up compressing my CSS files to ".css.gz" and keeping a copy of the original .css file in the same directory, I then use the following mod_rewrite code:

AddType text/css css gz
AddEncoding gzip gz

RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{REQUEST_URI} ^(.*).css
RewriteCond %{SCRIPT_FILENAME}.gz -f
RewriteRule ^(.*).css $1.css.gz

But I shouldn't have to do this, if mod_deflate just worked. My PHP generated output is fine, as I can just enable compression in my php.ini, it's just the CSS I really wanted to compress, but must use this other method for now I guess.

Really considering swithing to Ubuntu very soon

Submitted by Anonymous (not registered) on Thu, 2006-06-08 07:43.

Nice tutorial, thanks.

It may be useful to note, that also broadband access users profit from zipped content. Their browsing experience is getting snappier as well.

The reason is as follows:

Gzipped content is smaller. It is likely it fits into fewer tcp packages than non-zipped content. So you save packages but spend more time unzipping the content. When unzipping is faster then the delay until the next tcp package arrives, then you gain time and browsing becomes faster. With today's computers, unzipping is generally faster than the network delay, so zipping web content pays almost all the time for the browser.


As pointed out by Falko, on the serverside, you trade of CPU for memory disguised as network usage. This means, that non-zipped content means more tcp packages, means longer connection to the user, means using an apache process/thread for a longer time, means more processes forked, means more memory used.

In most webserver situations you have lots of CPU, but not enough memory. That's why zipping content helps in most situations on the server too.

Submitted by Anonymous (not registered) on Wed, 2006-05-17 10:02.
This just worked without problems.
Thanks for helping me saving bandwidth :)
Submitted by mevans (registered user) on Wed, 2008-03-12 01:39.

This approach minimizes number of http requests and file size while using little cpu resources:

1) Concatenate all your css and js files int a single file and then minify them.  Do it with ant tasks.  Here is a good tutorial: http://www.julienlecomte.net/blog/2007/09/16/

2) Serve css and js files from pre-compiled  files (I used .gz for css files and  .jgz for js files). Serve dynamic content from mod_deflate.  

3) Test using YUI whyslow for firefox 

The config for #2 is below: 
 

AddType text/css css gz
AddEncoding gzip gz
AddType text/javascript js jgz
AddEncoding gzip jgz


RewriteEngine on

RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !.*^Mozilla/4.* [OR]
RewriteCond %{HTTP_USER_AGENT} .*MSI?E.*
RewriteRule ^(.*)concated.min.css $1concated.min.css.gz

RewriteCond %{HTTP:Accept-encoding} gzip
RewriteCond %{HTTP_USER_AGENT} !.*^Mozilla/4.* [OR]
RewriteCond %{HTTP_USER_AGENT} .*MSI?E.*
RewriteRule ^(.*)concated.min.js $1concated.min.js.jgz



AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript text/css

# Netscape 4.x has some problems...
BrowserMatch ^Mozilla/4 no-gzip

# MSIE masquerades as Netscape, but it is fine
 BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

# NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
# the above regex won't work. You can use the following
# workaround to get the desired effect:
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html


# Make sure proxies don't deliver the wrong content
#Header append Vary User-Agent env=!dont-vary

Submitted by Mikhailov Anatoly (not registered) on Tue, 2008-12-16 13:49.
Article about mod_deflate settings like on Amazon EC2 AMI
http://railsgeek.com/2008/12/16/apache2-httpd-improving-performance-mod_deflate-gzip