In this series, the focus has been on the basic Web performance concepts, the ones that have dominated the performance management field for the last decade. It’s now time to step beyond these measures, and examine two equally important concepts, ones that allow a company to analyze their Web performance outside the constraints of performance and availability.

Reliability is often confused with availability when it is used in a Web performance context. Reliability, as a measurement and analysis concept goes far beyond the binary 0 or 1 that the term availability limits us to, and places it in the context of how availability affects the whole business.

Typical measures used in reliability include:

• Minutes of outage
• Number of failed measurements
• Core business hours

Reliability is, by its very nature, a more complex way to examine the successful delivery of content to customers. It forces the business side of a company to define what times of day and days of the week affect the bottom-line more, and forces the technology side of the business to be able to account not simply for server uptime, but also for exact measures of when and why customers could not reach the site.

This approach almost always leads to the creation of a whole new metric, one that is uniquely tied to the expectations and demands of the business it was developed in. It may also force organizations to focus on key components of their online business, if a trend of repeated outages appears with only a few components of the Web site.

Consistency is uniquely paired with Reliability, in that it extends the concept of performance to beyond simple aggregates, and considers what the performance experience is like for the customer on each visit. Can a customer say that the site always responds the same way, or do you hear that sometimes your site is slow and unusable? Why is the performance of your site inconsistent?

A simple way to think of consistency is the old standby of the Standard Deviation. This gives the range in which the population of the measurements is clustered around the Arithmetic Mean. This value can depend on the number of measures in the population, as well as the properties of these unique measures.

Standard Deviation has a number of flaws, but provides a simple way to define consistency: a large standard deviation value indicates a high degree of inconsistency within the measurement population, whereas a low small standard deviation value indicates a higher degree of consistency.

The metric that is produced for consistency differs from the reliability metric in that it will always be measured in seconds or milliseconds. But the same insight may arise from consistency, that certain components of the Web site contribute more to the inconsistency of a Web transaction. Isolating these elements outside the context of the entire business process gives organizations the information they need to eliminate these issues more quickly.

Companies that have found that simple performance and availability metrics constrain their ability to accurately describe the performance of their Web site need to examine ways to integrate a formula for calculating Reliability, and a measure of Consistency into their performance management regime.

Tags: analysis, Consistency, HTTP, measurement, performance, Reliability, Web Performance

I submitted a presentation proposal for OSCON 2005 just now. The abstract is below.

The Open Source community has driven the online world for the last decade. PHP, PERL, Apache, Java, and MySQL are all major components of large online enterprises.

However, putting an application online and ensuring that it satisfies the performance, availability and reliability demands of the increasingly knowledgeable online consumer are often two separate concerns.

Performance should not be an afterthought; performance should be a leading force in creating a Web application.

Using simple Open Source Tools, Web performance measurement solutions can be built that rival commercial solutions. But what does this data tell you? And how do you turn this into useful business information?

This discussion will expose the participants to key Web performance metrics that make sense to both technology and business leaders in your organization.

I have a snowball’s chance in hell of having it accepted, as it is not hip, technical or trendy, and I am not an Open Source Guru, but if you design stuff for the Web, then you better be ready to have your site examined in detail, because if you don’t do it, your customers will.

Tags: apache, app, application, business, community, components, customers, data, design, EAD, HTTP, IE, IM, information, IP, it, measurement, metrics, mysql, Om, One, online, online world, OReilly, performance, performance measurement, Perl, PHP, pr, Reliability, web, web app, web application, Web Performance, Web performance measurement

In writing the last post, I was thinking about what factors go into making the Web performance of a site "excellent". What defines in the minds of the sites users/customers/visitors/critics/competitors that the performance of a Web site is excellent?

These are usually judged by the standard factors:

• Usability
• Responsiveness
• Availability
• Traffic
• Reliability
• Security
• Clarity

But within the company itself, how is the performance of their Web site judged to be excellent?

Right now, most people use the external metrics mentioned above to determine excellence. However, it must be remembered that there are two other critical factors that need to be considered when managing a large IT infrastructure.

• Ease of Management. This is a metric that is often overlooked when determining if a Web site is excellent from an internal perspective. Often it is simply assumed that running a large IT infrastructure is incredibly complex; in most cases this is true. However, is it too complex too manage efficiently and effectively? How much time is spent finding the cause of problems as compared to resolving them?
• Cost of Operation. This is always a big one with me. I look at sites that are trying to squeeze as much performance and availability out of their sites as they can. At some point, the business has to step back and ask, "How much does another half-second of speed cost us?". When this context is placed around the "need for speed", it may open a few eyes.

When this two critical internal factors are combined with the raw external data that can be collected, collated and analyzed, some other ideas come to the forefront as KPIs in Web Performance Excellence:

• Cost Per Second. The cost of a Web site is usually only calculated based on the negative metric of how much it costs when the site is down. Well, how much does it cost when the site is up? Can that number be reduced?
• Revenue By Speed. Which customers spend the most on your site: LAN, home-broadband, or dial-up?
• Person-hours per day. How many person-hours per day does it take to manage your Web site?
• True Cost of Performance Issues. When there is a performance issue, the cost is usually associated with lost revenue. Reverse that and ask how much did it cost in time and materials to resolve the issue.

The creation of new Web performance excellence metrics is crucial if companies truly want to succeed in the e-business arena. Business management has to demand that IT management become more accountable to the entire business, using metrics that clearly display the true cost of doing business on the Web.

Tags: broadband, business, critical, customers, data, IE, IM, ISP, Issues, it, LAN, metrics, Om, One, performance, performance issues, pr, Reliability, run, traffic, users, visitors, web, Web Performance

I gave up. Blogger is too slow, flaky and clunky. I would rather pay the folks at TypePad/Six Apart.

Once again, I walk away from the free Google service into a for-fee service: Yahoo! Mail Plus and TypePad.

What does everyone else think?

ADDENDUM: Just checked in with my one other contact who uses Blogger, and she stated that she is seeing the same level of spotty performance and reliability that I have been experiencing. Her interaction with the Blogger support team resulted in an admission that the back-end at Blogger was getting some more kit to try and remedy these issues.

Too late, was the cry.

Tags: blog, Google, IE, interaction, IP, Issues, it, Om, One, performance, Reliability, Yahoo!