How much improvement can you see with compression? The difference in measured download times on a very lightly loaded server indicates that the time to download the Base Page (the initial HTML file) improved by between 1.3 and 1.6 seconds across a very slow connection when compression was used.

Base Page Performance

There is a slightly slower time for the server to respond to a client requesting a compressed page. Measurements show that the median response time for the server averaged 0.23 seconds for the uncompressed page and 0.27 seconds for the compressed page. However, most Web server administrators should be willing to accept a 0.04 increase in response time to achieve a 1.5 second improvement in file transfer time.

First Byte Performance

Web pages are not completely HTML. How do improved HTML (and CSS) download times affect overall performance? The graph below shows that overall download times for the test page were 1 to 1.5 seconds better when the HTML files were compressed.

Total Page Performance

To further emphasize the value of compression, I ran a test on a Web server to see what the average compression ratio would be when requesting a very large number of files. As well, I wanted to determine what the affect on server response time would be when requesting large numbers of compressed files simultaneously. There were 1952 HTML files in the test directory and I checked the results using CURL across my local LAN.[1]

Large sample of File Requests (1952 HTML Files)

mod_gzip

mod_deflate[2]

Average Compression by File Size mod_gzip mod_deflate 0-999 0.713 0.777[3] 1000-4999 0.440 0.440 5000-9999 0.389 0.389 10000-19999 0.369 0.369 20000-49999 0.350 0.350 50000 and up 0.329 0.331

Conclusion

With a few simple commands, and a little bit of configuration, an Apache Web server can be configured to deliver a large amount of content in a compressed format. These benefits are not simply limited to static pages; dynamic pages generated by PHP and other dynamic content generators can be compressed by using the Apache compression modules. When added other performance tuning mechanisms and appropriate server-side caching rules, these modules can substantially reduce the bandwidth for a very low cost.

[1] The files were the top level HTML files from the Linux Documentation Project. They were installed on an Apache 1.3.27 server running mod_gzip and an Apache 2.0.44 server using mod_deflate. Minimum file size was 80 bytes and maximum file size was 99419 bytes.

[2] mod_deflate for Apache/2.0.44 and earlier comes with the compression ratio set for Best Speed, not Best Compression. This configuration can be modified using the tips found here; and starting with Apache/2.0.45, there will be a configuration directive that will allow admins to configure the compression ratio that they want.

In this example, the compression ratio was set to Level 6.

[3] mod_deflate does not have a lower bound for file size, so it attempts to compress files that are too small to benefit from compression. This results in files smaller than approximately 120 bytes becoming larger when processed by mod_deflate.

A little-used or discussed feature of PHP is the ability to compress output from the scripts using GZIP for more efficient transfer to requesting clients. By automatically detecting the ability of the requesting clients to accept and interpret GZIP encoded HTML, PHP4 can decrease the size of files transferred to the client by 60% to 80%.

The information given here is known to work on systems running Red Hat 8.0, Apache/1.3.27, Apache/2.0.44 and PHP/4.3.1.

[Note: Although not re-tested since this article was originally written, compression is still present in the PHP 5.x releases and can be used to effectively compress content on shared or hosted servers where compression is not enabled within the Web server.]

Configuring PHP

The configuration needed to make this work is simple. Check your installed Red Hat RPMS for the following two packages:

1. zlib
2. zlib-devel

For those not familiar with zlib, it is a highly efficient, open-source compression library. This library is used by PHP uses to compress the output sent to the client.

Compile PHP4 with your favourite ./configure statement. I use the following:

Apache/1.3.27
./configure –without-mysql –with-apxs=/usr/local/apache/bin/apxs –with-zlib

Apache/2.0.44
./configure –without-mysql –with-apxs2=/usr/local/apache2/bin/apxs –with-zlib

After doing make && make install, PHP4 should be ready to go as a dynamic Apache module. Now, you have to make some modifications to the php.ini file. This is usually found in /usr/local/lib, but if it’s not there, don’t panic; you will find some php.ini* files in the directory where you unpacked PHP4. Simply copy one of those to /usr/local/lib and rename it php.ini.

Within php.ini, some modifications need to be made to switch on the GZIP compression detection and encoding. There are two methods to do this.

Method 1:

output_buffering = On
output_handler = ob_gzhandler
zlib.output_compression = Off

Method 2:

output_buffering = Off
output_handler =
zlib.output_compression = On

The ob_gzhandler

The most important component of this entire process is placing the ob_gzhandler PHP command on the page itself. It needs to be placed in the code at the top of the page, above the HTML tag in order to work. It takes the addition of the following line to complete the process:

<?php ob_start("ob_gzhandler"); ?>

In Wordpress installs, this becomes the first line in the HEADER.PHP file. But be careful to check that it’s working properly. If the Web application has the compression function built into it, and you add the ob_gzhandler function, a funky error message will appear at the top of the page telling you that your can’t invoke compression twice.

Web servers with native compression are smarter than that – they realize that the file is already compressed and don’t run it through the compression algorithm again.

Once this is in place, you will be able to verify the decrease in size using any HTTP browser capture tool (Firebug, Safari Web Inspector, Fiddler2, etc.)

So?

The winning situation here is that for an expenditure of $0 (except your time) and a tiny bit more server overhead (you’re probably still using fewer resources than if you were running ASP on IIS!), you will now be sending much smaller, dynamically generated html documents to your clients, reducing your bandwidth usage and the amount of time it takes to download the files.

How much of a size reduction is achieved? Well, I ran a test on my Web server, using WGET to retrieve the file. The configuration and results of the test are listed below.

Method 0: No Compression wget www.pierzchala.com/resume.php	File Size: 9415 bytes
Method 1: ob_gzhandler wget –header=”Accept-Encoding: gzip,*” www.pierzchala.com/resume.php	File Size: 3529 bytes
Method 2: zlib.output_compression wget –header=”Accept-Encoding: gzip,*” www.pierzchala.com/resume.php	File Size: 3584 bytes

You will have to experiment with the method that give the most efficient balance between file size and overhead and processing time on your server.

A 62% reduction in transferred file size without affecting the quality of the data sent to the client is a pretty good return for 10 minutes of work. I recommend including this procedure in all of your future PHP4 builds.

Ok, one of the tests that the GrabPERF System is running is doing a simple search on the Technorati site.

Ouch.

Now, as I mentioned before, Technorati has some interesting things going on in their www servers. For the Web geeks out there, here is what their headers look like.

HTTP/1.0 200 OK
Date: Mon, 11 Jul 2005 16:15:47 GMT
Server: Apache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
X-Powered-By: PHP/4.3.11
Set-Cookie: TECHNORATI_MEMBER=deleted; \
                   expires=Sun, 11 Jul 2004 16:15:46 GMT; path=/
Set-Cookie: TECHNORATI_MEMBER=[blah-blah]; \
                   expires=Wed, 12 Oct 2005 16:15:47 GMT; path=/; \
                   domain=technorati.com
Pragma: no-cache
Content-Type: text/html; charset=UTF-8
Connection: close

The Cache-Control header I can understand. The Pragma: no-cache is basically useless, as the Pragma header is only considered as a valid client-side header.

But the use of HTTP/1.0 and the explicit Connection: close really bother me. Removing the ability for clients to maintain persistent connections is a Network and Server resource issue, and should be avoided at all costs.

And where’s the compression? There is no need to send raw content to clients that understand and can process compressing text content. Using either hardware or mod_gzip or mod_deflate, the size of transferred content can be very easily reduced.

Sorry for the rant, but I seem to be on a Web performance kick lately. And the team at Technorati is one that I would expect to have gotten the need for this Web performance thing.

Technorati: web performance, HTTP, Technorati

HTTP Compression is a well-acknowledged way to improve Web performance and decrease bandwidth usage by compressing text content before transmitting it to the client. This has become an increasingly interesting topic for Web 2.0 sites starting to experience their first growth pains.

The sample above is far from representative. However, I would have thought that companies leading the Web 2.0 revolution would be learning the Web performance lessons of the previous generation.

All data was gathered using the GrabPERF Monitoring System

Technorati: web performance, HTTP, GrabPERF System, Internet, Web 2.0

Web page compression is not a new technology, but it has just recently gained higher recognition in the minds of IT administrators and managers because of the rapid ROI it generates. Compression extensions exist for most of the major Web server platforms, but in this article I will focus on the Apache and mod_gzip solution.

The idea behind GZIP-encoding documents is very straightforward. Take a file that is to be transmitted to a Web client, and send a compressed version of the data, rather than the raw file as it exists on the filesystem. Depending on the size of the file, the compressed version can run anywhere from 50% to 20% of the original file size.

In Apache, this can be achieved using a couple of different methods. Content Negotiation, which requires that two separate sets of HTML files be generated — one for clients that can handle GZIP-encoding, and one for those who can’t — is one method. The problem with this solution should be readily apparent: there is no provision in this methodology for GZIP-encoding dynamically-generated pages.

The more graceful solution for administrators who want to add GZIP-encoding to Apache is the use of mod_gzip. I consider it one of the overlooked gems for designing a high-performance Web server. Using this module, configured file types — based on file extension or MIME type — will be compressed using GZIP-encoding after they have been processed by all of Apache’s other modules, and before they are sent to the client. The compressed data that is generated reduces the number of bytes transferred to the client, without any loss in the structure or content of the original, uncompressed document.

mod_gzip can be compiled into Apache as either a static or dynamic module; I have chosen to compile it as a dynamic module in my own server (more compile instructions here). The advantage of using mod_gzip is that this method requires that nothing be done on the client side to make it work. All current browsers — Mozilla, Opera, and even Internet Explorer — understand and can process GZIP-encoded text content.

On the server side, all the server or site administrator has to do is compile the module, edit the appropriate configuration directives that were added to the httpd.conf file, enable the module in the httpd.conf file, and restart the server. In less than 10 minutes, you can be serving static and dynamic content using GZIP-encoding without the need to maintain multiple codebases for clients that can or cannot accept GZIP-encoded documents.

When a request is received from a client, Apache determines if mod_gzip should be invoked by noting if the “Accept-Encoding: gzip” HTTP request header has been sent by the client. If the client sends the header, mod_gzip will automatically compress the output of all configured file types when sending them to the client.

This client header announces to Apache that the client will understand files that have been GZIP-encoded. mod_gzip then processes the outgoing content and includes the following server response headers.

		Content-Type: text/html
		Content-Encoding: gzip
		

These server response headers announce that the content returned from the server is GZIP-encoded, but that when the content is expanded by the client application, it should be treated as a standard HTML file. Not only is this successful for static HTML files, but this can be applied to pages that contain dynamic elements, such as those produced by Server-Side Includes (SSI), PHP, and other dynamic page generation methods. You can also use it to compress your Cascading Stylesheets (CSS) and plain text files. As well, a whole range of application file types can be compressed and sent to clients. My httpd.conf file sets the following configuration for the file types handled by mod_gzip:

		mod_gzip_item_include mime ^text/.*
		mod_gzip_item_include mime ^application/postscript$
		mod_gzip_item_include mime ^application/ms.*$
		mod_gzip_item_include mime ^application/vnd.*$
		mod_gzip_item_exclude mime ^application/x-javascript$
		mod_gzip_item_exclude mime ^image/.*$
		

This allows Microsoft Office and Postscript files to be GZIP-encoded, while not affecting PDF files. PDF files should not be GZIP-encoded, as they are already compressed in their native format, and compressing them leads to issues when attempting to display the files in Adobe Acrobat Reader.^[1] For the paranoid system administrator, you may want to explicitly exclude PDF files.

		mod_gzip_item_exclude mime ^application/pdf$
		

Another side-note is that nothing needs to be done to allow the GZIP-encoding of OpenOffice (and presumably, StarOffice) documents. Their MIME-type is already set to text-plain, allowing them to be covered by one of the default rules.

How beneficial is sending GZIP-encoded content? In some simple tests I ran on my Web server using WGET, GZIP-encoded documents showed that even on a small Web server, there is the potential to produce a substantial savings in bandwidth usage.

Server administrators may be concerned that mod_gzip will place a heavy burden on their systems as files are compressed on the fly. I argue against that, pointing out that this does not seem to concern the administrators of Slashdot, one of the busiest Web servers on the Internet, who use mod_gzip in their very high-traffic environment.

The mod_gzip project page for Apache 1.3.x is located at SourceForge. The Apache 2.0.x version is available from here.

[1] From http://www.15seconds.com/issue/020314.htm