Helping a colleague this week, we uncovered some odd behavior with a site whose performance he was analyzing. Upon first glance, it was clear that this site had a performance issue – they had HTTP persistence disabled. Immediate red flag in the areas of network overhead and geographic latency.

Further digging exposed something more sinister. It seems that HTTP persistence was only disabled for browsers with MSIE in the user-agent string. Even if the user-agent string was just MSIE, HTTP persistence was off.

The customer was very forthcoming and sent us their standard httpd.conf file. This showed no sign of the standard (and frustrating) global disabling of persistence for Internet Explorer.

Finally, it came to us. The customer had provided a simple network diagram, and there, just before packets hit the Internet, was a Layer 7 firewall. How did we know the Layer 7 firewall was the likely cause? Because this device was also the one that provided compression for the content going out to customers.

A Layer 7 firewall happily rewrites HTTP headers to reflect the nature of the compressed content (content-length or transfer-encoding: chunked) and to add the gzip flag (accept-encoding:gzip). Since this device was already doing this, it was pretty clear to us that it also had a rule that disabled HTTP persistence for anything with MSIE in the user-agent string.

This was a fine example of the complexity of the modern Web application infrastructure. In effect, there were two groups with different ideas of how Internet Explorer should be handled at the network layer, and neither of them seems to have talked to the other.

When you have a Web performance problem, indulge in a thought experiment. Create an imaginary incoming Web request and try to see if you can follow it through all the systems it touches on your system. Put it on a whiteboard, a mindmap, whatever works.

Then invite the system architects and network engineers in and get them to fill in the gaps.

No doubt that will lead to the “ah ha!” moment. If nothing else, it’s a good excuse to put pizza on the company card. But I have no doubt that you will walk away with a better understanding of your systems, which will make it easier for you to talk to all the people responsible for keeping your systems running.

TAKEAWAY: Just because the part of the Web application you work on is working fine, it may be affected by other components that are not tuned or configured for performance. Get to know the entire application at a high level.

The title is a question I ask because I hear so many different views and perspectives about HTTP compression from the people I work with, colleagues and customers alike.

There appears to be no absolute statement about the compression capabilities of all current (or in-use) browsers anywhere on the Web.

My standard line is: If your customers are using modern browsers, compress all text content — HTML (dynamic and static), CSS, XML, and Javascript. If you find that a subset of your customers have challenges with compression (I suggest using a cross-browser testing tool to determine this before your customers do), write very explicit regular expressions into your Web server or compression device configuration to filter the user-agent string in a targeted, not a global, way.

For example, last week I was on a call with a customer and they disabled compression for all versions of Internet Explorer 6, as the Windows XP pre-SP2 version (which they say you could not easily identify) did not handle it well. My immediate response (in my head, not out loud) was that if you had customers using Window XP pre-SP2, those machines were likely pwned by the Russian Mob. I find it very odd that an organization would disable HTTP compression for all Internet Explorer 6 visitors for the benefit of a very small number of ancient Windows XP installations.

Feedback from readers, experts, and browser manufacturers that would allow me to compile a list of compatible browsers, and any known issues or restrictions with browsers, would go a long way to resolving this ongoing debate.

UPDATE: Aaron Peters pointed me in the direction of BrowserScope which has an extensive (exhaustive?) list of browsers and their capabilities. If you are seeking the final word, this is a good place to start, as it tests real browsers being used by real people in the real world.

Once again it is time to analyze browser usage in the US for the last month. July saw the appearance of Firefox 3.5, which has replicated the pattern seen with Internet Explorer 8, where it supplants the previous version slowly and linearly as people get around to upgrading.

Can MSIE 8 overtake MSIE 7 in August? How much will Firefox 3.5 usage grow in August and will it replace FF 3.0 as the dominant version in the Firefox family?

As with previous analyses, Internet Explorer 6 retains its iron grip on the corporate, custom Web application market. The question is not when, but if, this browser will actually fade away. It is unlikely that Internet Explorer 6 will disappear until Windows 2000 and Windows XP percentages are in serious decline.

This points to a larger concern that organizations will have to face within the next 18 months: What do they do when the Windows 2000 lifecycle terminates in July 2010 and as Windows XP sees fewer updates moving toward lifecycle retirement in 2014? [See the Microsoft LifeCycle information here]

Hiding from the inevitable just makes changes that much more dramatic and difficult.

It is not likely that the patterns in the StatCounter data will change until the summer vacation season is over in the US, and students bring their shiny new computers online at the start of the school year.

June is one of my favorite months. The sun returns (although in the Boston area there are concerned that it has been replaced by clouds and humidity), the kids get out of school (and get sent to camp), and the outdoor pool opens (and I actually swim in it).

In the US browser market, Internet Explorer 8 continues its slow replacement of Internet Explorer 6 and 7, finally overtaking MSIE 6 on June 11 [Stats courtesy of StatCounter].

The great news is that Internet Explorer 6 is slowly falling of the pace, relegated to large companies with proprietary code and a degree of inertia that impedes their movement to accepting new browsers.

The two-month trend does show some very dramatic changes, most notably with Internet Explorer 7 and Firefox 3.

While these changes appear dramatic, the lack of absolute values to base the StatCounter graphs on means that it’s very difficult to determine if these values are a result of a shift in the actual browser market, or a result in decreasing numbers of visitors to sites with the StatCounter tracking code.

Worldwide for June, the primary trend is that the decrease in Internet Explorer 7 is matched almost precisely by the increase in the use of Internet Explorer 8.

Firefox 3 and Internet Explorer 6 remained almost completely unchanged through June, indicating that the US trend is very different than that seen throughout the rest of the world. The tracking trend indicates that Firefox 3 could have overtaken Internet Explorer 7 by the end of July.

Could have is used purposely here, as the release of Firefox 3.5 will fragment the market share for this browser, and it is not likely that it will match the stats for Firefox 3 immediately.

Despite all the claims that the browser war is over, and that applications have moved beyond the browser, it is highly unlikely that this dream will be realized in the consumer browser market until late 2010, when the effect of Windows 7 can be seen on the use of Internet Explorer 8 .

Overall, June 2009 was a month of substantial shifts in the US browser market, which will be further aggravated with the release of Firefox 3.5, and the slow and steady adoption of Internet Explorer 8 by consumer and business users.

UPDATE: TechCrunch has noted the ongoing shifts to the browser share market [here].

This week marks a momentous time in the history of the Internet. In the United States, StatCounter reports that for the first three days of the work week (Monday – Friday), Internet Explorer 8 usage is equal to Internet Explorer 6 usage.

Tie this to the trend of decreasing Internet Explorer usage noted late last week and the release of Firefox 3.5 RC1 and Safari 4.0 and Opera 10 in the last few weeks, and it appears that the balance of browser usage on the Internet is becoming more fluid.

Does this mean that standards will become more relevant? Can you truly count on limiting customers to one browser?

Will browser lock-in for certain applications continue to be considered acceptable?

Since its GA release on March 19 and its addition to Windows Update in late April, Internet Explorer 8 has been gradually increasing its market share in the US. Based on the current growth pattern in StatCounter’s GlobalStats data, it appears that Internet Explorer 8 will overtake Internet Explorer 6 sometime in late May or early June.

In other parts of the world, the adoption of the new version of Internet Explorer is substantially slower, and affected by regional differences in the browser population. Europe is notable in this as MSIE8 has just overtaken Opera 9.6 in the browser population in the last week.

In Asia, Internet Explorer 8 has moved into fourth in browser share, but is a substantially lower percentage of the population than the top three browsers. In a frightening statistic, Internet Explorer 6 is the most popular browser in Asia, indicating that path to adoption may be longer in this region.

Overall the adoption of this new browser as a replacement for Internet Explorer 6 and 7 is slow and steady. MSIE8 does not appear to be significantly cutting into the Firefox population, but this could change as people begin to hear more about the features of the new browser, and Web sites begin to be designed to its features, rather than those of MSIE6 and MSIE7.

At CNet News today, the article What browser wars? The enterprise still loves IE 6 nicely sums all the reasons that Internet Explorer 6 is still in use in enterprise environments. The dominance of Internet Explorer 6 in the workplace is something I discussed a few days ago, supported by the pattern of higher weekday use of this browser during the work-week.

Limiting employees to a browser that is considered an ancient technology by Web developers poses an interesting dichotomy to companies. Internally, they are powering their internal applications with coal and steam, and turning the cogs of business with leather drive-belts. Externally, the customers get to see a site that is shiny, one that has all the gadgets of a rich-Internet application that require the computationally advanced capabilities of a modern browsers.

To some extent, the companies who use Internet Explorer 6 internally are saying to their employees that the Internet is simply another desktop application that they must use.

I am firm believer that the Web will be the home of many of the applications we use on a daily basis in the very near future [here and here]. My vision of this is that the Web application is designed to free the user, and lessen the workload for IT departments.

Most organizations continue to see the Internet as a negative, a threat to productivity, a necessary evil. So if they restrict their employees and provide them with a browser that doesn’t quite work properly on the Internet, they still have ultimate control.

I have had the benefit of working for organizations throughout my professional career that did (do) not limit my choice of browser or operating system, allowing me to find my own preference. I realize that this is rare in corporate Industrial Culture, and I consider myself lucky.

So, why is their still a need for Internet Explorer 6? Frankly, there is really no need for it in my opinion, and the CNet News article has a table from Forrester that supports that. That doesn’t mean that my browser idealism and utopian dream of “may the best browser win” will hold any sway over the IT decision makers in large organizations.

StatCounter still shows that during the week, Internet Explorer 6 holds 12% of the browser share in the US [here]. So, will this 12% of the browser world hold back the promise of the Internet for the rest of us?

While I was performing my standard Windows Update on my work virtual machine this morning, I wondered if the promised Internet Explorer 8.0 release to Windows Update had been dropped.

I switched to my test-bed, vanilla Windows XP virtual machine, ran Windows Update, and PING! Up it came. The masses of people who blindly do what Windows Update tells them to do will now be installing Internet Explorer 8 on their machines.

The interesting thing is that the only piece of software that Windows Update said was a critical Update was Internet Explorer 8.0. Given that it would be replacing Internet Explorer 7.0 on my virtual machine, how bad was Internet Explorer 7? Are they trying to push MSIE6 and MSIE7 out of the way ASAP?

Today should see a large number of new installs of Internet Explorer 8, either on purpose or inadvertently by those folks who install everything that Microsoft tells them to. I will be monitoring StatCounter’s GlobalStats over the next few days to see if there is a spike in, most notably, US installs of Internet Explorer 8.

Just as a sidebar, Internet Explorer 8, without the support of Windows Update, has increased from 3.5% to 6.28% of the browser share in the US (3.33% to 6.2% in North America; 2.65% to 4.4% Worldwide) in the April 1-28 2009 period. [Stats courtesy of StatCounter GlobalStats]

I can’t make a lot of comments about the quality of browsing experience in one version of Internet Explorer over another; I have been a dedicated Firefox user on Mac, and a Safari user on Windows (yes, that is weird) for a while. But the desire to move as many people as quickly as possible to a new browser speaks volumes to where Microsoft thinks the Web is going. And they realize that it is not going in the direction that its older browsers had been taking it.

Further Updates will follow.

UPDATE: A colleague forwarded me this article on the release of Internet Explorer 8 to Windows Update. Effectively, you still have to go through the Web interface and agree to download the new browser. It’s not being pushed down onto Windows users through the automatic update built into the OS, no doubt to placate the glacially-slow IT departments I mentioned here.

Internet Explorer 6.0, that infamous dinosaur from the dark ages of 2001, is still with us. And on occasion, I have hinted that this is the result of the biblically-slow pace of change in large corporate IT departments.

Well, now I have proof of this.

Using data from our good friends at StatCounter, this graph leaped from the screen and nearly exploded my mind.

The pattern in the data is clear, if you are paying attention, which I haven’t been. The dips that appear in the Internet Explorer stats occur on weekends.

That means that many of the Internet Explorer 6.0 users are only using it because it is forced on them by their IT departments.

Why?

Dear IT professionals of the world: Internet Explorer 6.0 is an exploit waiting to happen and a barrier to inventive Web development. Please upgrade. Thank you.

Tomorrow morning, at 09:00 PDT, Microsoft unleashes IE 8.0 on an unsuspecting Internet.

By 12:00 PDT, the number of Web sites having to put up IE 7.0 only stickers will be in the millions.

I haven’t done a lot of testing of the new monster (I mainly use Firefox on Mac OS 10.5.6), but it doesn’t seem any weirder than any of the other Microsoft browsers I have used.

Under the hood, it does amp up the number of connections that are used to download objects, but if your infrastructure is prepared, it should be no big deal.

The big concern lies in the rendering. If those users who can upgrade to the new browser (anyone not limited to IE 6 by a corporate IT policy written on parchment) do find there are problems, will they blame the site or the browser?

Have you tested your site to ensure that it actually works with IE 8? Or will you get burned?

What is the greatest benefit you see to your visitors having IE 8 instead of IE 6 or 7?

Is this a good thing?