In the Customer Generation and Customer Retention articles of this series, the focus was on Web performance measurements designed to serve an audience outside of your organization. Starting with Business Operations, the focus shifts toward the use of Web performance measurements inside your organization.

Why Business Operations?

When I was initially developing these ideas with my colleague Jean Campbell, the idea was to call this section Reporting and Quality of Service. What we found was that this didn’t completely encompass all of the ideas that fall under these measurements. The question became: which part of the organization do reporting and QoS measurements serve?

What was clear was these were the metrics that reported on the health of the Web service to management and the company as a whole. This was the measurement data that the line of business tied to revenue and analytics data to get a true picture of the health of the online business.

What are you measuring?

Measurements for business operations need to capture the key metrics that are critical for making informed business decisions.

• How do we compare to our competitors?
• Are we close to breaching our SLAs?
• Are the third-parties we use close to breaching their SLAs?
• What parts of the site affect performance / user experience the most so we can set priorities?
• How does Web performance correlate with all the other data we use in our online business?

Every company will use different measures to capture this information, and correlate the data in different ways. The key is that you do use it to understand how Web performance ties into the line of business.

How often do I look at it?

Well, honestly, most people who work in business operations only need to examine Web performance once a day in a summary business KPI report (your company has a useful daily KPI report that everyone understands and uses, right?), and in greater detail at weekly and monthly management meetings.

The goal of the people examining business operations data is not to solve the technical problems that are being encountered, but to understand how the performance of their site affects the general business health of the company, and how it plays in the competitive marketplace.

What metrics do I need?

Business operations teams need to understand

• End-to-end response time for measured business processes
• Page-level response times for measured business processes
• Success rate of the transaction during the measurement period
• How third-parties are affecting performance
• How Web analytics and Web performance relate
• How different regions are affected by performance
• How does performance look from the customer ISPs and desktops

Detailed technical data is lost on these people. It is their role to take all of the data they have, and present a picture of the application as it affects the business, and discuss challenges that they face at a technical level in terms of how they affect the business.

Summary

For people who work at an extremely detailed level with Web measurement data (the topic for the next part of this series), Business Operations metrics seem light, fluffy, and often meaningless. But these metrics serve a distinct audience: the people who run the company. Frankly, if the senior business leaders at an organization are worried on a daily basis about the minute technical details taht go into troubleshooting and diagnosing performance issues, I would be concerned.

The objective of Business Operations measurements is to convey the health of the Web systems that support the business, and correlate that health with other KPIs used by the management team.

Tags: baselining, benchmarking, business leaders, business operations, comparative metrics, correlation, Jean Campbell, KPI, line of business, online, Quality of Service, senior management, SLA, SLO, Web analytics, Web measurement data, Web Performance, Web performance measurements, Web performance ties, Web service, Web systems

This week, I have been discussing the different approaches to blog analytics that can be used to determine what posts from a blog’s archive are most popular, and whether a blog is front-loaded or long-tailed. The thesis is that it’s not always what the words in the blog are that are important.

In a guest post this morning at ProBlogger, Skellie discusses how the value of social media visitors is different and inherently more complex than the value of visitors generated from traditional methods, such as search and feedreaders. Her eight points further support my ideas that the old advertising models are not the best suited for the new blogging world.

Stepping away from the existing advertising models that have been used since blogging popularity exploded in 2005 and 2006, it is clear that the new, interactive social web model requires an advertising approach that centers on community and conversation, rather than the older idea of context and aggregated readership.

The Current Model

Current blog advertising falls into two categories:

1. Contextual Ads. This is the Google model, and is based on the ad network auctioning off keywords and phrases to advertisers for the privilege of seeing their ad links or images appear on pages that contain those words or phrases.
2. Sponsored Ads. Once a blog is popular enough and can prove a well-developed audience, the blogger can offer to sell space on his blog to advertisers who wish to have their products, offerings or companies presented to the target audience.

In my opinion, these two approaches fail blog owners.

Contextual ads understand the content of the page, but do not understand the popularity of the page, or its relationship to the popularity of other pages in the archive.Contextual ads lack a sense of community, a sense of conversation. While the model has proven successful, it does not maximize the reach that a blog has with its own audience.

Sponsored ads understand the audience that the blog reaches, but do not account for posts that draw the readers’ attention for the longest time, both in terms of time spent reading and thinking about the post as well as over time in an historical sense. The sponsored ad model assumes that all posts get equal attention, or drive community and conversation to the same degree.

The New Model

In the new model, more effective use of visitor analytics is vital to shaping the type and value of the ads sold. Studying the visitor statistics of a blog will allow the owners to see whether the blog is, in general, front-loaded or long-tailed.

If the blog has a front-loaded audience, the most recent posts are of higher value and could be auctioned of at higher prices. In order for this to work, both the ad-hoster and the advertiser would have to agree to the value of the most recent posts using a proven and open statistical analysis methodology. In the case of front-loaded blogs, this analysis methodology would have to demonstrate that there is a higher traffic volume for posts that are between 0-3 days old (setting a hypothetical boundary on front-loading).

For blogs that are long-tailed, those posts that continue to draw consistent traffic would be valued far more highly than those that fall out into the general ebb and flow of a bloggers traffic. These posts have proven historically that they appear highly in search results and are visited often.

In addition to the posts themselves, the comment stream has to be considered. Posts that generate an active conversation are farmore valuable those that don’t. Again, showing the value of the conversation is reliant of the ability to track the numbers of people in the conversation (through Disqus or some other commenting system).

This model can be further augmented by using a tool like Lookery that helps to clearly establish the demographics of the blog audience. Being able to pinpoint not only where on a blog to advertise but also who the visitors are who view those page, provides a further selling point for this new model and helps build faith in the virtues of a blog that sells space using this new, more effectively targeted advertising pricing structure.

Now, I separate the front-loaded and long-tailed blogs as if they are distinct. Obviously these categories apply to nearly every blog as there are new posts that suddenly capture the imagination of an audience, and there are older posts that continue to provide specific information that draws a steady stream of traffic to them.

Summary

This is a very early stage idea, one that has no code or methodology to support it. However, I believe that the current contextual advertising model, one based solely on the content of the post, is not allowing the content creators and blog entities to take advantage of their most valuable resource - their own posts and the conversations that they create.

I also believe that blog owners are not taking advantage of their own best resource, Web analytics, to help determine the price for advertising of their site. Not all blog posts are created or read equally. Being able to very clearly show what drives the most eyeballs to your site is a selling point that can be used in a variable-price advertising model.

By providing tools to blog owners that intimately link the analytics they already gather and the advertising space they have to sell, a new advertising model can arise, one that is uniquely suited to the new Web. This advertising model will be founded in the concepts of conversation and community, providing more discretely targeted eyeballs to advertisers, and higher ad revenues to blog owners and content creators.

UPDATES

Appears that BuzzLogic has already started down this path. VentureBeat has commentary here.

Tags: ad network, advertising, advertising approach, advertising initiatives, advertising model, advertising models, advertising space, analytics, audience, blog advertising, blog advertising works, BuzzLogic, contextual ads, contextual advertising, contextual advertising model, front-loaded, Google, interactive social web model, long tail, long-tailed, search results, social media visitors, statistics, traffic, variable-price advertising model, VentureBeat, views, Web analytics

This afternoon, StatCounter showed a marked increase in performance.

Normally I wouldn’t highlight an issue that only lasted an hour, but this appears to have been a very unusual issue that saw the page size decrease to nearly nothing, and performance shoot up to around 45 seconds. This combination usually indicates a back-end application timeout which then presents users with an error message.

StatCounter is in the GrabPERF Site Statistics Index.

Technorati Tags: StatCounter, Web Performance, GrabPERF, Web Metrics, Web Analytics

Tags: analytics, app, application, ARIN, GrabPERF, GrabPERF.org, HTTP, IE, IM, it, metrics, normal, Om, performance, PHP, pr, statistics, Technorati, users, web, Web analytics, Web Performance

The Site Statistics | Web Analytics Index measurements have been running now for about 2.5 days, and I wanted to make some general comments on what I am seeing.

The methodolgy for testing is straightforward. I chose sites | services that allowed you to create a free (if limited) account to track your Web visitors, and allowed you to make these statistics available to for anyone to look at. Using this this, a measurement was established against the landing page that visitors would see if they chose to look at these publicly available statistics.

I am using this blog as the placeholder for the tracking “bugs”  used in this index (see the right-hand column).

From the graph above, it is clear that ShinyStat is the performance leader in this space. They have the smallest overall page size as well as the fastest and most reliable performance.

It is important to note that services such as WebTrends, Omniture, WebSideStory and Coremetrics are not included, as they are beyond the reach of most bloggers, and do not provide a public side to their data. Also, Google Analytics is not included, as they do not provide public access to the collected data.

The collected data is available in GrabPERF as both the Site Statistics Index, and as individual measurements.

Technorati Tags: Web analytics, web metrics, shinystat, onestat, statcounter, sitemeter, GrabPERF, Web Performance

Tags: analytics, ARIN, blog, bugs, data, EAD, Google, GrabPERF, GrabPERF.org, HTTP, IE, IM, it, LAN, measurement, metrics, Om, One, performance, PHP, pr, run, statistics, Technorati, testing, tracking, visitors, web, Web analytics, Web Performance

I have now launched the GrabPERF Site Statistics Index. The current members are:

• StatCounter
• SiteMeter
• ShinyStats
• OneStat

If you know of or represent a firm that I have missed, please drop me an email or a comment.

Technorati Tags: GrabPERF, Web performance, StatCounter, SiteMeter, OneStat, ShinyStats, Site Statistics, Web Analytics, performance index

Tags: analytics, current, GrabPERF, GrabPERF.org, HTTP, IE, it, Om, One, performance, PHP, pr, statistics, Technorati, web, Web analytics, Web Performance

You’ll notice that I have added the SiteMeter and StatCounter logos to the sidebar. Yes, I am a stats freak, but I also wanted to monitor how quickly each of these two services can deliver a page of basic stats to someone who clicks on the links.

So far, you can see the performance difference here.

I know that there are other services out there, and if you can think of one you use that i have forgotten, drop an e-mail or a comment. I want to create a site analytics benchmark, and also do some analysis of the differences in values captured by each of the services.

UPDATE [17:31 GMT]: I added OneStat to the list. New graph is here.

UPDATE [18:01 GMT]: I added ShinyStat to the list. New graph is here.

Technorati Tags: Web analytics, hit tracking, visitor tracking

Tags: analysis, analytics, GrabPERF, GrabPERF.org, HTTP, it, live, Om, One, performance, PHP, Technorati, tracking, update, web, Web analytics

Google drops price for Urchin Web Analytics. [here and here and here and here]

StatCounter gives me what I need, thanks.

Tags: analytics, ASP, Google, HTML, HTTP, IM, marketing, Om, PHP, pr, traffic, web, Web analytics

Back in November, I mentioned that I was working on the idea of new ways to benchmark the success of online businesses in today’s more mature operational environment. I am still working on the base ideas, but a colleague of mine has helped me coalesce some ideas, and they are now forming the foundation of the concepts my company will begin using internally to more deeply understand the various Web performance benchmarks we monitor.

For those who use the existing Web performance benchmarks to determine the success and failure of your online business, you understand how thin the veneer is on these benchmarks. They do not provide true insight into the operational success of an online business, and they are more likely to sow the seeds of distrust between IT and Business operations in the long-term by creating an artificial standard which becomes the goal.

If an online business truly wants to achieve and maintain exemplary Web performance numbers, it has to start with a strong foundation, and build on it. Why? The team I work with spends a lot of time trying to understand and reverse engineer the broken processes, designs, and architectures that were laid out in order to get big fast. After 3-4 years of technical starvation and underfunding, these online businesses are beginning to show strain; the temporary fix has become the permanently broken process.

The rush into the Web analytics space in the last few weeks is a key sign that companies now see value in and want to exploit the vast quantities of data that they collect on their traffic daily. Web analytics is an astoundingly complex field, but most people boil it up to a single concept: How many Unique Page Views did I get?

Unique Page Views is an outdated Web server analytics metric. It does not tell me anything about the business, other than it has a lot of traffic. Back in the “eyeballs are everything” period, this would have been a big deal. Now, I say so what, and start asking:

• From where
• Dialup? Broadband?
• How many were able to successfully complete their transactions?
• What paths are most visited
• Average spend by connection type?
• Average spend by hour?
• etc.

Like Unique Page Views, the average Web performance and availability of a Web page or transaction does not accurately represent the overall health of any online business. Within the large populations of data that exist at the Web measurement firms, there is a wealth of data that could be used to clearly expose more important benchmarking statistics.

If you are from an online business, you already understand that the average performance over an artificially-defined period of time is a very inaccurate way to measure the success of the online business. However, it is the accepted standard in the field. Underlying those aggregated values, there are clearly-defined statistical methods which can be used to extract even more meaningful information from the mass of measurement data.

I would discuss more of the ideas and concepts that we are working on, but I know that I do get visitors from our competitors, so I will have to keep our ideas under wraps for right now.

But I want to hear yours. What does your online business use as a benchmark for success? Standard avergae performance and availability? Or something more complex that examines the performance data as a complete population, as opposed to an aggregated summary value? Does your firm tie business goals and objectives into the performance benchmarks so that people across the company can understand how the business is succeeding, and how delivering a good, bad, and downright awful online performance experience can affect the bottom line?

This is an exciting time to have access to large amounts of data on the health of the Internet.

Tags: analytics, benchmarking, benchmarks, broadband, business, cro, data, design, EAD, HTTP, IE, IM, information, Internet, it, live, maintain, measurement, Om, One, online, page views, performance, PHP, pr, server, statistics, traffic, views, visitors, web, Web analytics, Web Performance, Web server

NetIQ sold off WebTrends to Francisco Partners. [here]

I always wondered about the NetIQ deal, so it’s nice to see WebTrends on it’s own again…but it is now going up against Google.

More on the deals here and here

More from SVW here.

John Battelle With more comments on Web Analytics Shake-Up 2005. [here]

Andrew Goodman has a great Urchin/Google Round-Up. [here]

Tags: analytics, ASP, blog, Google, HTML, HTTP, IE, it, media, Om, PHP, tn, traffic, web, Web analytics

Ouch. I thought this one was worthy of some analysis. Google acquire Urchin. [and here]

Why does this make sense? As one of the comments to the link above states, the link between search algorithms makes some sense, as you will be able quickly isolate and identify trends within the reams of visitor data that is collected daily.

This makes linking business performance and visitor traffic that much easier for large enterprise firms. If they can link visitor traffic quickly to revenue and cost data with a simple and powerful search tool, the sale becomes that much easier.

I am working on a project to re-define Web performance benchmarks, and this type of tool would be crucial in that process. Web Benchmarking as it is currently stands is broken and outdated. Attempts are being made on all sides to try and re-define how businesses measure and benchmark their Web performance so that it can fit into a larger business context.

Is a faster site more profitable? Or simply more costly? Does increased traffic improve or reduce revenue?

A tool that allow companies to quickly link performance and analytics data gets firms that much closer to a holistic view of Web performance.

Tags: analysis, analytics, benchmarking, benchmarks, blog, business, current, data, Google, HTTP, IE, IM, it, media, Om, One, performance, PHP, pr, search, traffic, web, Web analytics, Web Performance