Dear Apache Software Foundation, and the developers of the Apache Web server:

I would like to thank you for developing a great product. I rely on it daily to host my own sites, and a large number of people on the Internet seem to share my love of this software.

However, it appears that you seem to want to maintain a simple flaw in your logic that continues to make me crazy. I am a Web performance analyst, and at least once a week I sigh, and shake my head whenever I stoop to use Microsoft Internet Explorer (MSIE) to visit secure sites.

I seems that in your SSL configurations, you continue to assume that ALL versions of MSIE can’t handle persistent connections under SSL/TLS.

Is this true? Is a bug initially caught in MSIE 5.x (5.0??) still valid for MSIE 6.0/7.0?

The short answer is: I don’t know.

It seems that no one in the Apache server team has bothered to go back and see if the current versions of MSIE — we are trying to track down the last three people use MSIE 5.x and help them — still share this problem.

In the meantime, can you change your SSL exclusion RegEx to something more, relevant for 2007?

Current RegEx:

SetEnvIf User-Agent ".*MSIE.*" nokeepalive
	ssl-unclean-shutdown
	downgrade-1.0 force-response-1.0

Relvant, updated REGEX:

SetEnvIf User-Agent ".*MSIE [1-5].*"
	nokeepalive ssl-unclean-shutdown
	downgrade-1.0 force-response-1.0

SetEnvIf User-Agent ".*MSIE [6-9].*"
	ssl-unclean-shutdown

Please? PLEASE? It’s so easy…and would solve so many performance problems…

Please?

Thank you.

Tags: Apache, SSL, TLS, MSIE, persistent connections, keepalive, web performance

Tags: apache, Dear Apache Software Foundation, http persistence, Internet Explorer, Microsoft, persistent, ssl, Web Performance, Web server

This morning’s bounty of posts brought in two that will make you think.

First was Port80 Software’s comments on using the Cache-Control mechanism embedded in all browsers. This is interesting to read, as I have been trying to get companies to use this mechanism more intelligently for a number of years. I know that the Port80 team gets it, but it is always nice to have some outside validation of a position you have tried to evangelize for a long time.

The second was Tim O’Reilly’s post on Cal Henderson’s new book on Web scalability. While I am likely to purchase the book for a professional interest, I have one problem with Flickr’s current configuration: static.flickr.com does not use HTTP persistence, something I noted last week. This strikes me as weird.

It’s always good to see Web performance rear its head.

Technorati Tags: Tim O’reilly, Cal Henderson, Cache-Control, http, port80 software, persistent connections, Flickr

Tags: ASP, book, browser, cache-control, control, current, EAD, HTML, HTTP, http persistence, IE, IM, it, Om, One, OReilly, performance, persistent, persistent connections, PHP, posts, pr, Technorati, web, Web Performance

Dear Flickr:

I have been wondering for sometime why downloads from your site seemed a little sluggish at times.

At first I blamed your unprecedented growth and success. For a little Vancouver startup (I am a BC boy myself), your entrance onto the stage of social networking applications has been phenomenal. The move from zero to infinity may have played a part in the performance I was seeing.

Nope. There was something else going on; I could see it every time I loaded a Flickr page in my browser. There was something else going on.

So today, I checked something out, and found the problem.

You need to enable persistent TCP connection on the static.flickr.com servers.

Now, that is the simple answer. I know that with large, web-based applications, enabling something as monumental as persistent connections could cause serious issues. If the architecture of the system was not designed to handle persistent connections, turning them on could cause the entire system to collapse.

There are legitimate, if mis-guided, reasons for disabled persistent connections. Some administrators believe that it is actually more efficient to have a client open a connection for every object. Easier to manage state, etc. The only problem is that in order to do that, you have to tune the systems serving data to shorten the amount of time a closed connection spends in a TIME_WAIT state.

When a TCP connection is closed, the socket is not immediately closed by the system in a default configuration. The TIME_WAIT state is the holding pen that these connections are pushed into. While in this state, the socket is locked and this may count against the incoming TCP connection queue, forcing the network stack to delay or reset new incoming connections.

Still, as Flickr is a worldwide company, the delay that the lack of persistent connections injects is astounding for locations in Asia. If you want to grow your business, and support more services, this will likely become a bottleneck very quickly.

Have a great weekend!

smp

Technorati Tags: Flickr, persistent connections, TCP, http, web performance, scalability

Tags: ads, app, application, Asia, browser, business, client, data, design, HTTP, IE, IM, Issues, it, media, networking, Om, performance, persistent, persistent connections, pr, server, startup, TCP, Technorati, web, Web Performance, web-based application

After a few month hiatus, I am starting to code for GrabPERF again. I need to exercise my brain; as I am a hobbyist code mangler, I have to take on a project every now and then to keep my not-so-l33t skillz honed.

The change to the agent is one of efficiency. The current production agent opens two database connections to run tests: one to retrieve the test configuration data; the other to insert the results of the tests. This means I loop through one set of database query results while doing inserts inside the loop on a second database connection.

This is stupid.

The new code opens a single connection to the database, retrieves the test configuration, dumps the results to an array of arrays, then inserts the data on the same connection. This is more efficient, as I use persistent connections and compression to MySQL to improve performance.

I have this running as TEST AGENT 1 from the Technorati #2 site.

Let me know if you see any madness…outside of Washington DC, and specifically with GrabPERF.

Technorati Tags: GrabPERF, PERL, MySQL

Tags: compress, compression, current, data, database, GrabPERF, GrabPERF.org, HTTP, IE, IM, it, mysql, Om, One, performance, Perl, persistent, persistent connections, pr, run, Technorati, testing, the other, Washington DC

One of the issues with not being a coder is that development tends to be kludegy. Add in SQL queries and the mess gets worse.

I have made some mods to the Index graphs (daily and hourly) that should only penalize the first person in one of my persistent database connections. After that, the queries are cached.

And while sitting here, I just realized I could improve the speed of this process even more.

I. Am. An. Idiot.

I will fix it….later.

Tags: data, database, GrabPERF, IE, IM, Issues, it, Om, One, performance, persistent, pr

Ok, one of the tests that the GrabPERF System is running is doing a simple search on the Technorati site.

Ouch.

Now, as I mentioned before, Technorati has some interesting things going on in their www servers. For the Web geeks out there, here is what their headers look like.

HTTP/1.0 200 OK
Date: Mon, 11 Jul 2005 16:15:47 GMT
Server: Apache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
X-Powered-By: PHP/4.3.11
Set-Cookie: TECHNORATI_MEMBER=deleted; \
                   expires=Sun, 11 Jul 2004 16:15:46 GMT; path=/
Set-Cookie: TECHNORATI_MEMBER=[blah-blah]; \
                   expires=Wed, 12 Oct 2005 16:15:47 GMT; path=/; \
                   domain=technorati.com
Pragma: no-cache
Content-Type: text/html; charset=UTF-8
Connection: close

The Cache-Control header I can understand. The Pragma: no-cache is basically useless, as the Pragma header is only considered as a valid client-side header.

But the use of HTTP/1.0 and the explicit Connection: close really bother me. Removing the ability for clients to maintain persistent connections is a Network and Server resource issue, and should be avoided at all costs.

And where’s the compression? There is no need to send raw content to clients that understand and can process compressing text content. Using either hardware or mod_gzip or mod_deflate, the size of transferred content can be very easily reduced.

Sorry for the rant, but I seem to be on a Web performance kick lately. And the team at Technorati is one that I would expect to have gotten the need for this Web performance thing.

Technorati: web performance, HTTP, Technorati

Tags: apache, cache-control, client, client-side, compress, compressing, compression, content, control, deflate, EAD, GrabPERF, gzip, HTML, HTTP, IE, IM, IP, it, LAN, maintain, media, mod_gzip, Om, One, performance, persistent, persistent connections, PHP, pr, run, search, server, Technorati, web, Web Performance, Web+performance

You have noted that you are experiencing some performance issues related to high load (here). So I investigated and found that all the servers at the hostname www.technorati.com are responding with HTTP/1.0 headers and are explicitly closing the connections to the clients.

Why?

This will not relieve the performance problems. In fact, doing this may make the situation worse. The only way that this will not cause an issue is if you have tuned the kernel on these servers to go through an EXTREMELY fast TCP teardown process on the server side.

If you haven’t done this, go run a netstat -vanp on your www servers. See all the fin_wait, fin_wait2, closing and time_wait states? These are sockets that can’t be used again until the kernel releases them.

Now, Turning on keep-alives or persistent connections on the www servers will:

1. Reduce the total number of connections per client
2. Reduce the number of sockets “hung” in the teardown process
3. Improve performance by reducing the network overhead required by the client and the server

The only caveat is to reduce the keep-alive timeout to something like 4-5 seconds so that these connections don’t wait for traffic forever.

Improving performance through disabling persistent connections is a myth. HTTP/1.1 was adopted for a reason. Use it wisely and your will reap the rewards.

Technorati: web performance, HTTP, Technorati, Apache, PHP

Tags: apache, blog, client, EAD, HTML, HTTP, IE, IM, Issues, it, live, Om, One, performance, performance issues, persistent, persistent connections, PHP, pr, run, server, sockets, TCP, Technorati, tn, traffic, web, Web Performance, Web+performance

I have been seeing these bursts of traffic, mainly from spambot morons, that have suddenly been crushing my server. The main cause: excessive database connections.

This was quickly remedied today when I changed all of the mysql_connect statements to mysql_pconnect statements. This allows PHP to use an existing connection to the MySQL database to serve requests from the same Apache child process.

Now the truly geeky among you are going “DOH! Wadda ya mean you were opening a new connection for every request?”. Well, believe it or not, I will bet you dollars to doughnuts that your blog app doesn’t persist database connections. Not a big deal if your database is on the same machine, and you are using local named pipes to make requests. However, if that database is located on another machine, if you do a netstat, you will see a large number of connection on port 3306.

Persisting database connections is particularly important for large hosted services. A great deal of TCP overhead, and kernel space memory can be saved by simply not letting the Web server saturate the database with individual database connections for every page request.

Without persistent database connections, eventually the TCP queue will be full of database connections and no one will be able to connect to the server, or they will get a lovely “can’t connect to database error”.

Technorati: a href=”http://www.technorati.com/tag/web+performance” rel=”tag”>web performance, PHP, MySQL, MySQL PHP, Apache

Tags: apache, app, blog, data, database, EAD, HTML, HTTP, IE, IM, IP, it, Mac, mysql, Om, One, performance, persistent, PHP, pr, server, spam, TCP, Technorati, traffic, web, Web Performance, Web server, Web+performance