A decade of working in the Web performance industry can leave one with the idea that no matter how good a site is, there is always the opportunity to be better, be faster. However, I am beginning to believe, just from my personal experience on the Internet, that speed has reached its peak with the current technologies we have.

This does not bode well for an Internet that is shifting more directly to true read/write, data/interaction heavy Web sites. This needs to have home broadband that is not only fast, but which has equality for inbound and outbound connection speeds.

But will faster home broadband really make that much of a difference? Or will faster networks just show that even with the best connectivity to the Internet money can buy, Web sites are actually hurting themselves with poor design and inefficient data interaction designs?

For companies on the edge of Web performance, who are trying to push their ability to improve the customer experience as hard as possible, who are moving hard and fast to the read/write web, here are some ways you can ensure that you can still deliver the customer experience your vistors expect.

Confirm your customers’ bandwidth

This is pretty easy. Most reasonably powerful Web analytics tools can confirm this for you, breaking it down by dialup, and high broadband type. It’s a great way to ensure that your preconceptions about how your customers interact with your Web site meets the reality of their world.

It is also a way to see just how unbalanced your customers’ inbound and outbound connection speeds. If it is clear that traffic is coming from connection types or broadband providers that are heavily weighted towards download, then optimization exercises cannot ignore the effect of data uploads on the customer experience.

Design for customers’ bandwidth

Now that you’ve confirmed the structure of your customers’ bandwidth, ensure that your site and data interaction design are designed with this in mind. Data that uses a number of inefficient data calls behind the scenes in order to be more AJAXy may hurt itself when it tries to make those calls over a network that’s optimized for download and not upload.

Measure from the customer perspective

Web performance measurement has been around a long time. But understanding how the site performs from the perspective of true (not simulated) customer connectivity, right where they live and work, will highlight how your optimizations may or may not be working as expected.

Measurements from high-throughput, high-quality datacenter connections give you some insight into performance under the best possible circumstances. Measure from the customer’s desktop, and even the most thoughtfully planned optimization efforts may have been like attacking a mammoth with a closed safety pin: ineffective and it annoys the mammoth [to paraphrase Hugh Macleod].

As well as synthetic measurements, measure performance right from within the browser. Understanding how long it takes pages to render, how long it takes to show content above the fold, and to gather discrete times on complex Flash and AJAX events within the page will give you even more control over finding those things you can fix.

Takeaway

In the end, even assuming your customers have the best connectivity, and you have taken all the necessary precautions to get Web performance right, don’t assume that the technology can save you from bad design and slow applications.

Be constantly vigilant. And measure everything.

My Google Alerts today picked up a post from a former colleague of mine, commenting on another post from the Yahoo! Interface blog.

I had some problems following the stream in the Performance Matters post, so I thought I would this post to clear up my thoughts.

A technical note up front: Using a waterfall chart that only shows non-persistent connections gives a very skewed view on how a modern Web page page performs. Persistent connections and modern TCP/IP stacks with fast-retransmit algorithms and window-scaling have seend a trend away from network-related performance issues in the recent past.

After the dot-bomb crash, the overabundance of bandwidth (in the form of over-built fiber-optic networks) made backbone and end-to-end connectivity issues for business and most home broadband users almost completely disappear.

The wealth of bandwidth (ok, North America consumers arethe poor cousins compared to their European and Asian counterparts) removed the veil of “it’s the network” which had been the crutch of performance engineers for many years, and exposed the effects of poor design and badly designed infrastructure.

In many cases, poor page design could be overcome. However, issues with core infrastructure and application design were (and are) notoriously difficult to resolve without spending a lot of money and investing a large amount of time and manpower.

So, when these issues were combined with the shrinking budgets and constricted IT staffing in the post-boom era, application performance issues became (and still are) the root-cause of most Web performance issues.

In recent months, as the use of Internet telephony, rich-media streaming, file-sharing, RSS, and SOA products rise nearly exponentially, the bandwidth crunch is starting to re-appear. This is something I first speculated about in October 2005 [here].

In one area, I do agree with the Performance Matters post: the larger the page, the slower download. However, the ongoing debate is one that pits the “more smaller” crown against the “fewer larger” crowd. The “fewer larger” crowd appears to be losing, given the design of most modern Web pages.

The only other comment I can fairly make here is that the majority of the sources cited in the Performance Matters post are 5-6 years old. In that time, I have learned a lot about Web performance, and that the post is more relevant to to the state of the Internet at that time, and not now.

This post has two underlying reasons for existing: 1) to test out the new MSFT Live Writer Beta; and 2) to talk about a great story that GigaOm pointed us to today.

Om Malik pointed out a story in the Seattle Times today that talked about “Broadband in the Boonies”. Having grown up in the boonies of British Columbia, this immediately got my attention. The story discusses the explosive growth of Internet businesses in the now heavily wired interior of Washington State; the story focuses on the are around Twisp, Winthrop and the Methow Valley.

Until you have been in this area, and I have, you don’t get the possibility of winter isolation. The story talks about how these places are four hours from Seattle; what they neglect to mention is that this is 4 hours in the period between April 15 and October 15, depending on snow.

The direct westerly route to Seattle from these locations passes through the Cascades. Through the extremely high and snowy Cascades.

Samantha and I took a spur of the moment detour through this little part of heaven, pausing a night in a campground in Twisp. Right on the river. When we woke up the next morning, I remembered how much I missed those early morning moments in the mountains.

Twisp is far more isolated than Golden, BC, or any of the other towns that we passed through on our trip this summer. But it is a reminder to us all that place is important. Not because we have to be there, but because it is where we are at home.

I have lived in the Valley. I have lived in Massachusetts. But neither has been home.

And to me, home is worth more than anything.

Technorati Tags: GigaOm, Seattle+Times, Twisp, Winthrop, Methow+Valley, Washington+State, Home, Broadband, Freedom, Om+Malik

Ping-O-Matic. Weblogs.com.

Two of the sites that have seen surges in Web traffic recently.

My unsubstantiated theory is that as summer ends, Web traffic is seeing its usual surge, amplified by the new interest in blogs and Web 2.0 properties.

For the small companies who have effectively had the summer off, it’s time to re-adjust your capacity-planning estimates…upward.

Why? Rising oil prices will reduce travel. Disenchantment with movies will reduce going out. Gourmet take-home will reduce restaurant visitation. And increasing broadband access will increase demand for online services.

Be afraid…and get ready for more traffic. Lots more traffic.

Have you optimized your site for maximum performance yet?

Technorati: Web Performance, Capacity Planning, Web 2.0, SEO, traffic analysis

IceRocket: Web Performance, Capacity Planning, Web 2.0, SEO, traffic analysis

Ok folks, now that GrabPERF has been up and running for a few weeks, and people have actually been looking at the data (including the CEO’s of two blog search tools!), it’s time to ask the community that is using this service to contribute to keep it running.

Decommissioned hardware. $20. I am not asking for the world.

Currently, this system runs on my home broadband connection using six year-old hardware. that is…well, having issues.

For all of you who do contribute, I thank you in advance.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, blog search

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, blog search

Ok folks, now that GrabPERF has been up and running for a few weeks, and people have actually been looking at the data (including the CEO’s of two blog search tools!), it’s time to ask the community that is using this service to contribute to keep it running.

Decommissioned hardware. $20. I am not asking for the world.

Currently, this system runs on my home broadband connection using six year-old hardware. that is…well, having issues.

For all of you who do contribute, I thank you in advance.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, blog search

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, blog search

Now, I never even heard about this Comcast outage until this morning. [here]

One reason that I likely didn’t notice it is that I run my own caching name servers on my home network. I do this because I have never trusted any provider’s overworked name servers (goes back to the history of the industry I work in; more on that some other time).

This is still a fairly crippling outage. Surely there is some redundancy in the Comcast name server infrastructure…please?

There is a list of possible workarounds to this problem linked from here.

Joi Ito, at the ICANN meetings in Brazil, has posted some interesting comments and links regarding the exhaustion of the IPv4 Space. [here]

The growth in private IP Spaces has helped stave off this exhaustion, even through the massive growth in the home broadband and mobile markets. I forsee a compromise coming shortly where IPv6 is used between Network Transit Providers, while IPv4 is reserved exclusively for customer-facing usage.

Just another of my hare-brained ideas…

Back in November, I mentioned that I was working on the idea of new ways to benchmark the success of online businesses in today’s more mature operational environment. I am still working on the base ideas, but a colleague of mine has helped me coalesce some ideas, and they are now forming the foundation of the concepts my company will begin using internally to more deeply understand the various Web performance benchmarks we monitor.

For those who use the existing Web performance benchmarks to determine the success and failure of your online business, you understand how thin the veneer is on these benchmarks. They do not provide true insight into the operational success of an online business, and they are more likely to sow the seeds of distrust between IT and Business operations in the long-term by creating an artificial standard which becomes the goal.

If an online business truly wants to achieve and maintain exemplary Web performance numbers, it has to start with a strong foundation, and build on it. Why? The team I work with spends a lot of time trying to understand and reverse engineer the broken processes, designs, and architectures that were laid out in order to get big fast. After 3-4 years of technical starvation and underfunding, these online businesses are beginning to show strain; the temporary fix has become the permanently broken process.

The rush into the Web analytics space in the last few weeks is a key sign that companies now see value in and want to exploit the vast quantities of data that they collect on their traffic daily. Web analytics is an astoundingly complex field, but most people boil it up to a single concept: How many Unique Page Views did I get?

Unique Page Views is an outdated Web server analytics metric. It does not tell me anything about the business, other than it has a lot of traffic. Back in the “eyeballs are everything” period, this would have been a big deal. Now, I say so what, and start asking:

• From where
• Dialup? Broadband?
• How many were able to successfully complete their transactions?
• What paths are most visited
• Average spend by connection type?
• Average spend by hour?
• etc.

Like Unique Page Views, the average Web performance and availability of a Web page or transaction does not accurately represent the overall health of any online business. Within the large populations of data that exist at the Web measurement firms, there is a wealth of data that could be used to clearly expose more important benchmarking statistics.

If you are from an online business, you already understand that the average performance over an artificially-defined period of time is a very inaccurate way to measure the success of the online business. However, it is the accepted standard in the field. Underlying those aggregated values, there are clearly-defined statistical methods which can be used to extract even more meaningful information from the mass of measurement data.

I would discuss more of the ideas and concepts that we are working on, but I know that I do get visitors from our competitors, so I will have to keep our ideas under wraps for right now.

But I want to hear yours. What does your online business use as a benchmark for success? Standard avergae performance and availability? Or something more complex that examines the performance data as a complete population, as opposed to an aggregated summary value? Does your firm tie business goals and objectives into the performance benchmarks so that people across the company can understand how the business is succeeding, and how delivering a good, bad, and downright awful online performance experience can affect the bottom line?

This is an exciting time to have access to large amounts of data on the health of the Internet.

1. I should not be running servers on my Comcast connection. BAD DOG! BAD DAMNATION HOUND!
2. Comcast should not make it so easy to spot, and then not be able to explain, a daily outage pattern

The nice lady in customer phone support said there were no problems in my area, but that she could not contact my modem. Hmmm….

Can you spot the pattern? Anyone have any recommendations for reliable hosting providers in MA? Or better yet, inexpensive and reliable home broadband options?