In writing the last post, I was thinking about what factors go into making the Web performance of a site "excellent". What defines in the minds of the sites users/customers/visitors/critics/competitors that the performance of a Web site is excellent?

These are usually judged by the standard factors:

• Usability
• Responsiveness
• Availability
• Traffic
• Reliability
• Security
• Clarity

But within the company itself, how is the performance of their Web site judged to be excellent?

Right now, most people use the external metrics mentioned above to determine excellence. However, it must be remembered that there are two other critical factors that need to be considered when managing a large IT infrastructure.

• Ease of Management. This is a metric that is often overlooked when determining if a Web site is excellent from an internal perspective. Often it is simply assumed that running a large IT infrastructure is incredibly complex; in most cases this is true. However, is it too complex too manage efficiently and effectively? How much time is spent finding the cause of problems as compared to resolving them?
• Cost of Operation. This is always a big one with me. I look at sites that are trying to squeeze as much performance and availability out of their sites as they can. At some point, the business has to step back and ask, "How much does another half-second of speed cost us?". When this context is placed around the "need for speed", it may open a few eyes.

When this two critical internal factors are combined with the raw external data that can be collected, collated and analyzed, some other ideas come to the forefront as KPIs in Web Performance Excellence:

• Cost Per Second. The cost of a Web site is usually only calculated based on the negative metric of how much it costs when the site is down. Well, how much does it cost when the site is up? Can that number be reduced?
• Revenue By Speed. Which customers spend the most on your site: LAN, home-broadband, or dial-up?
• Person-hours per day. How many person-hours per day does it take to manage your Web site?
• True Cost of Performance Issues. When there is a performance issue, the cost is usually associated with lost revenue. Reverse that and ask how much did it cost in time and materials to resolve the issue.

The creation of new Web performance excellence metrics is crucial if companies truly want to succeed in the e-business arena. Business management has to demand that IT management become more accountable to the entire business, using metrics that clearly display the true cost of doing business on the Web.

Tags: broadband, business, critical, customers, data, IE, IM, ISP, Issues, it, LAN, metrics, Om, One, performance, performance issues, pr, Reliability, run, traffic, users, visitors, web, Web Performance

Service Level Agreements (SLAs) appear to finally be maturing in the realm of Web performance. Both of the Web performance companies that I have worked for have understood their importance, but convincing the market of the importance of these metrics has been a challenge up until recently.

In the bandwidth and networking industries, SLAs have been a key component of contracts for many years. However, as Doug Kaye outlined in his book Strategies for Web Hosting and Managed Services, SLAs can also be useless.

The key to determining a useful Web performance SLA rests on some clear business concepts: relevance and enforceability. Many papers have been written on how to calculate SLAs, but that leaves companies still staggering with the understanding that they need SLAs, but don’t understand them.

Relevance

Relevance is a key SLA metric because an SLA defined by someone else may have no meaning to the types of metrics your business measures itself on. Whether the SLA is based on performance, availability or a weighted virtual metric designed specifically by the parties bound by the agreement, it has to mean something, and be meaningful.

The classic SLA is average performance of X seconds and availability of Y% over period Z. This is not particularly useful to businesses, as they have defines business metrics that they already use.

Take for example a stock trading company. in most cases, they are curious, but not concerned with their Web performance and availability between 17:00 and 08:00 Eastern Time.But when the markets are open, these metrics are critical to the business.

Now, try and take your stock-trading metric and overlay it at Amazon or eBay. Doesn’t fit. So, in a classic consultative fashion, SLAs have to be developed by asking what is useful to the client.

• Who is to be involved in the SLA process?
• How do SLAs for Internal Groups differ from those for External vendors?
• Will this be pure technical measurement? Will business data be factored in?

Asking and answering these questions makes the SLA definition process relevant to the Web performance objectives set by the organization.

Enforceability

The idea that an SLA with no teeth could exist is almost funny. But if you examine the majority of SLAs that are contracted between businesses in the Web performance space today, you will find that they are so vaguely defined and meaningless to the business objectives that actually enforcing the penalty clauses is next to impossible.

As real world experience shows, it is extremely difficult for most companies enforce SLAs. If the relevance objectives discussed above are hammered out so that the targets are clear and precise, then enforcement becomes a snap. The relevance objective often fails, because the SLA is imposed by one party on another; or an SLA is included in a contract as a feature, but when something goes wrong, escape path is clear for the “violating” party.

If an organization would like to try and develop a process to define enforceable SLAs, start with the internal business units. These are easier to develop, as everyone has a shared business objective, and all disputes can be arbitrated by internal executives or team leaders.

Once the internal teams understand and are able to live with the metrics used to measure the SLAs, then this can be extended to vendors. The important part of this extension is that third-party SLA measurement organizations will need to become involved in this process.

Some would say that I am tooting my own horn by advocating the use of these third-party measurement organizations, as I have worked for two of the leaders in this area. The need for a neutral third-party is crucial in this scenario; it would be like watching a soccer match (football for the enlightened among you) without the mediating influence of the referee.

---

If your organization is now considering implementing SLAs, then it is crucial that these agreements are relevant and enforceable. That way, both parties understand and will strive to meet easily measured and agreed upon goals, and understand that there are penalties for not delivering performance excellence.

Tags: app, bandwidth, book, business, client, critical, data, design, Doug Kaye, eBay, eBay Inc, extension, HTTP, IE, IM, ISP, it, leave, live, measurement, media, metrics, networking, performance, pr, SLA, third-party, web, Web Hosting, Web Performance, Web performance objectives, Web performance SLA, Web performance space