Newest Industry

Port80 Software is reporting that in their survey of Fortune 1000 Web sites, IIS 6.0 has overtaken Apache as the Web server platform of choice. [here]

My two-cents: I respect the Port80 Software team greatly and love their maniacal devotion to ensuring that IIS users actually make use of the HTTP compression and caching that can so greatly improve Web performance.

That said, they are tied to Microsoft and the IIS platform. I would be curious to see if, scratching below the surface, they were able to determine what the application platform these companies built their mission critical Web applications on. I am open-minded and willing to hear that IIS is winning in that area as well. In my mind, it’s about Web performance tuning, not what you use to get that performance.

That said, I think a critical Web application survey of these same firms would find that many of these companies rely on JSP servers to run their core business processes.

As well, it would be interesting to se, by Fortune 1000 ranking, what the companies are using what server platform.

And…people still use Netscape Enterprise, SunOne, and Domino as production Web servers? YIKES!

As a teenager growing up in a very small logging town in the BC interior, I had what could be politely termed unusual musical tastes, especially for the mainstream, heavy-metal, hair-banging kids I hung around with.

But when I was alone with my walkman, I listened to the real geniuses of 80s rock: REM, Kate Bush, Talking Heads, and…Lindsey Buckingham.

Lindsey Buckingham?? That guy from Fleetwood Mac?

Want a little aural treat? Listen to Go Insane. I literally wore the oxide off my version of the cassette. Crosses so many different boundaries…and realize that you are pretty much hearing Lindsey Buckingham only. Mick Fleetwood makes a couple appearances, but other than that, it is a one-man show.

Do it. I dare you.

Citizens Bank Outage

Originally uploaded by spierzchala.

Some days, your bank needs to get smacked around.

This is one of those times.

What is going on?

Ok Web performance gurus, I have been out-cooled by someone I work with. Ryan Breen, VP of Technology at Gomez and overall uber-geek, has managed to register AJAX Performance and has a blog up there that talks all about the freaky twisted goodness of making your AJAX behave.

Ryan knows way more about making apps behave; I just know how to analyze the data that shows that they’re broken.

Happy Thanksgiving

Originally uploaded by spierzchala.

To all the folks back home, Happy Thanksgiving!

May your turkey be moist, juicy, and, preferably, smoked.

Sometimes, you have to be in awe.

Today, they christened the Nimitz-class carrier, George H.W. Bush.

Still a few bugs to work out. Seems the navigation system breaks down after it has seen battle, causing it to wander aimlessly, and eventually become lost. It is especially vulnerable to attack by more than one enemy simultaneously, which in some simulations has forced the commander to surrender the vessel.

I also hear that they have started CAD drawings for the Seawolf-class nuclear submarine SSN George W Bush. Not only is it designed to be isolated from and out of contact with the rest of the world for long periods of time, I hear that it will have a new command feature: all fleet orders, battle information, or damage reports are first filtered through the boat’s Media Relations Officer before being passed to the commander.

File Under: Humor.

So, after 6 years of controlling and managing my own Web server, I have handed responsibility over to 1 & 1. I wish I could say that there was a really good reason why I’ve done this, but frankly, it’s because I don’t need a lot of oooommmmph for my personal domains (they run happily on a low-end Pentium II Celeron), and the price was right.

GrabPERF is still happily hosted by the folks at Technorati, while WordPress.com controls my blog.

In some ways, I am glad that someone else has these headaches now.

This paper is an extension of the work done for another article that highlighted the performance benefits of retrieving uncompressed and compressed objects directly from the origin server. I wanted to add a proxy server into the stream and determine if proxy servers helped improve the performance of object downloads, and by how much.

Using the same series of objects in the original compression article[1], the CURL tests were re-run 3 times:

1. Directly from the origin server
2. Through the proxy server, to load the files into cache
3. Through the proxy server, to avoid retrieving files from the origin.[2]

This series of three tests was repeated twice: once for the uncompressed files, and then for the compressed objects.[3]

As can be seen clearly in the plots below, compression caused web page download times to improve greatly, when the objects were retrieved from the source. However, the performance difference between compressed and uncompressed data all but disappears when retrieving objects from a proxy server on a corporate LAN.

Instead of the linear growth between object size and download time seen in both of the retrieval tests that used the origin server (Source and Proxy Load data), the Proxy Draw data clearly shows the benefits that accrue when a proxy server is added to a network to assist with serving HTTP traffic.

	MEAN DOWNLOAD TIME
Uncompressed Pages
Total Time Uncompressed — No Proxy	0.256
Total Time Uncompressed — Proxy Load	0.254
Total Time Uncompressed — Proxy Draw	0.110
Compressed Pages
Total Time Compressed — No Proxy	0.181
Total Time Compressed — Proxy Load	0.140
Total Time Compressed — Proxy Draw	0.104

The data above shows just how much of an improvement is gained by adding a local proxy server, explicit caching descriptions and compression can add to a Web site. For sites that do force a great of requests to be returned directly to the origin server, compression will be of great help in reducing bandwidth costs and improving performance. However, by allowing pages to be cached in local proxy servers, the difference between compressed and uncompressed pages vanishes.

Conclusion

Compression is a very good start when attempting to optimize performance. The addition of explicit caching messages in server responses which allow proxy servers to serve cached data to clients on remote local LANs can improve performance to even a greater extent than compression can. These two should be used together to improve the overall performance of Web sites.

---

[1]The test set was made up of the 1952 HTML files located in the top directory of the Linux Documentation Project HTML archive.

[2]All of the pages in these tests announced the following server response header indicating its cacheability:

Cache-Control: max-age=3600

[3]A note on the compressed files: all compression was performed dynamically by mod_gzip for Apache/1.3.27.

How much improvement can you see with compression? The difference in measured download times on a very lightly loaded server indicates that the time to download the Base Page (the initial HTML file) improved by between 1.3 and 1.6 seconds across a very slow connection when compression was used.

Base Page Performance

There is a slightly slower time for the server to respond to a client requesting a compressed page. Measurements show that the median response time for the server averaged 0.23 seconds for the uncompressed page and 0.27 seconds for the compressed page. However, most Web server administrators should be willing to accept a 0.04 increase in response time to achieve a 1.5 second improvement in file transfer time.

First Byte Performance

Web pages are not completely HTML. How do improved HTML (and CSS) download times affect overall performance? The graph below shows that overall download times for the test page were 1 to 1.5 seconds better when the HTML files were compressed.

Total Page Performance

To further emphasize the value of compression, I ran a test on a Web server to see what the average compression ratio would be when requesting a very large number of files. As well, I wanted to determine what the affect on server response time would be when requesting large numbers of compressed files simultaneously. There were 1952 HTML files in the test directory and I checked the results using CURL across my local LAN.[1]

---

Large sample of File Requests (1952 HTML Files)

mod_gzip

		Uncompressed	Compressed
First Byte
	Mean	0.091	0.084
	Median	0.030	0.036
Total Time
	Mean	0.280	0.128
	Median	0.173	0.079
Bytes per Page
	Mean	6349	2416
	Median	3750	1543
Total Bytes		12392318	4716160

mod_deflate[2]

		Uncompressed	Compressed
First Byte
	Mean	0.044	0.046
	Median	0.028	0.031
Total Time
	Mean	0.241	0.107
	Median	0.169	0.050
Bytes per Page
	Mean	6349	2418
	Median	3750	1544
Total Bytes		12392318	4720735

	mod_gzip	mod_deflate
Average Compression	0.433	0.438
Median Compression	0.427	0.427

---

As expected, the First Byte download time was slightly higher with the compressed files than it was with the uncompressed files. But this difference was in milliseconds, and is hardly worth mentioning in terms of on-the-fly compression. It is unlikely that any user, especially dial-up users, would notice this difference in performance.That the delivered data was transformed to 43% of the original file size should make any Web administrator sit up and notice. The compression ratio for the test files ranged from no compression for files that were less than 300 bytes, to 15% of original file size for two of the Linux SCSI Programming HOWTOs. Compression ratios do not increase in a linear fashion when compared to file size; rather, compression depends heaviliy on the repetition of content within a file to gain its greatest successes. The SCSI Programming HOWTOs have a great deal of repeated characters, making them ideal candidates for extreme compression.Smaller files also did not compress as well as larger files, exactly for this reason. Fewer bytes means a lower probability of repeated bytes, resulting in a lower compression ratio.

---

Average Compression by File Size mod_gzip mod_deflate 0-999 0.713 0.777[3] 1000-4999 0.440 0.440 5000-9999 0.389 0.389 10000-19999 0.369 0.369 20000-49999 0.350 0.350 50000 and up 0.329 0.331

---

The data shows that compression works best on files larger than 5000 bytes; after that size, average compression gains are smaller, unless a file has a large number of repeated characters. Some people argue that compressing files below a certain size is a wasteful use of CPU cycles. If you agree with these folks, using the 5000 byte value as floor value for compressing files should be a good starting point. I am of the opposite mindset: I compress everything that comes off my servers because I consider myself an HTTP overclocker, trying to squeeze every last bit of download performance out of the network.

Conclusion

With a few simple commands, and a little bit of configuration, an Apache Web server can be configured to deliver a large amount of content in a compressed format. These benefits are not simply limited to static pages; dynamic pages generated by PHP and other dynamic content generators can be compressed by using the Apache compression modules. When added other performance tuning mechanisms and appropriate server-side caching rules, these modules can substantially reduce the bandwidth for a very low cost.

---

[1] The files were the top level HTML files from the Linux Documentation Project. They were installed on an Apache 1.3.27 server running mod_gzip and an Apache 2.0.44 server using mod_deflate. Minimum file size was 80 bytes and maximum file size was 99419 bytes.

[2] mod_deflate for Apache/2.0.44 and earlier comes with the compression ratio set for Best Speed, not Best Compression. This configuration can be modified using the tips found here; and starting with Apache/2.0.45, there will be a configuration directive that will allow admins to configure the compression ratio that they want.

In this example, the compression ratio was set to Level 6.

[3] mod_deflate does not have a lower bound for file size, so it attempts to compress files that are too small to benefit from compression. This results in files smaller than approximately 120 bytes becoming larger when processed by mod_deflate.