There is clear dissatisfaction with the current state of marketing among the social media mavens.

• Fred Wilson and Union Square Ventures are looking for companies to invest in to take advantage of this.
• BuzzLogic releases their conversational ad service.
• The Inquisitr moves from AdSense to Technorati Media, indicating a potential shift at b5 Media.
• Lookery is providing demographic information that is relevant to social media providers and shaping ad presentation that way
• A completely off-the-radar person such as myself realizes that there is gold in what holds the attention, and that’s not always the latest and greatest

So what can be done? Jeff Jarvis points out that the problem lies with measurement. I agree, as there is only value in a system where all of the people involved agree on what the metric of record will be, and how it can be validly captured.

Currently CPM is the agreed upon metric. In a feed based online world, how does a CPM model work? And, most importantly, why would I continue to place your ads on my site if all your doing is advertising to people based on the words on the page, rather than who is looking at the page and how often that page is looked at.

In effect, advertisers should be the ones thrying to figure out how to get into the community, get into the conversation. As an advertiser, don’t you want to be where the action is? But how do you find an engaged audience in an online world that makes a sand castle on the beach in a hurricane look stable?

The challenge for advertisers is to be able to find the active communities and conversations effectively. The challenge for content creators and communities is to understand the value of their conversations, the interactions that people who visit the site have with the content.

In effect, a social media advertising model turns the current model on its head. Site owners and community creators gain the benefit of being attractive to advertisers because of the community, not because of the content. And site owners who understand who visits their site, what content most engages them, how they interact with the system will be able to reap the greatest rewards by selling their community as a marketable entity.

And Steven Hodson rounds out the week’s think on communities by throwing out the subversive idea that communities are not always free (as in ‘beer’, not as in ‘land of’). If a community has paid for the privilege of coming together to participate in communal events and discussions, then can’t that become an area for site owners to further control the cost of advertising on their site?

While the benefit of reduced or no marketing content is the benefit of many for-pay communities, this benefit can be used by site owners by saying that an advertiser can have access to the for-pay community at the cost of higher ad rates and smaller ads. The free community is a completely different set of rules, but there are also areas in the free community that are of higher value than others.

In summary, the current model is broken. But there is no way to measure the value of a Twitter stream, a FriendFeed conversation, a Disqus thread, or a Digg rampage. And until there is, we are stuck with an ad model that based on the words on the page, and not the community that created the words.

While this post is aimed at Web performance, the curse of the single metric affects our everyday lives in ways that we have become oblivious to.

When you listen to a business report, the stock market indices are an aggregated metric used to represent the performance of a set group of stocks.

When you read about economic indicators, these values are the aggregated representations of complex populations of data, collected from around the country, or the world.

Sport scores are the final tally of an event, but they may not always represent how well each team performed during the match.

The problem with single metrics lies in their simplicity. When a single metric is created, it usually attempts to factor in all of the possible and relevant data to produce an aggregated value that can represent a whole population of results.

These single metrics are then portrayed as a complete representation of this complex calculation. The presentation of this single metric is usually done in such a way that their compelling simplicity is accepted as the truth, rather than as a representation of a truth.

In the area of Web performance, organizations have fallen prey to this need for the compelling single metric. The need to represent a very complex process in terms that can be quickly absorbed and understand by as large a group of people as possible.

The single metrics most commonly found in the Web performance management field are performance (end-to-end response time of the tested business process) and availability (success rate of the tested business process). These numbers are then merged and transformed by data from a number of sources (external measurements, hit counts, conversions, internal server metrics, packet loss), and this information is bubbled up in an organization. By the time senior management and decision-makers receive the Web performance results, that are likely several steps removed from the raw measurement data.

An executive will tell you that information is a blessing, but only when it speeds, rather than hinders, the decision-making process. A Web performance consultant (such as myself) will tell that basing your decisions on a single metric that has been created out of a complex population of data is madness.

So, where does the middle-ground lie between the data wonks and the senior leaders? The rest of this post is dedicated to introducing a few of the metrics that will, in a small subset of metrics, give a senior leaders better information to work from when deciding what to do next.

A great place to start this process is to examine the percentile distribution of measurement results. Percentiles are known to anyone who has children. After a visit to the pediatrician, someone will likely state that “My son/daughter is in the XXth percentile of his/her age group for height/weight/tantrums/etc”. This means that XX% of the population of children that age, as recorded by pediatricians, report values at or below the same value for this same metric.

Percentiles are great for a population of results like Web performance measurement data. Using only a small set of values, anyone can quickly see how many visitors to a site could be experiencing poor performance.

If at the median (50th percentile), the measured business process is 3.0 seconds, this means that 50% of all of the measurements looked at are being completed in 3.0 seconds or less.

If the executive then looks up to the 90th percentile and sees that it’s at 16.0 seconds, it can be quickly determined that something very bad has happened to affect the response times collected for the 40% of the population between these two points. Immediately, everyone knows that for some reason, an unacceptable number of visitors are likely experiencing degraded and unpredictable performance when they visit the site.

A suggestion for enhancing averages with percentiles is to use the 90th percentile value as a trim ceiling for the average. Then side-by-side comparisons of the untrimmed and trimmed averages can be compared. For sites with a larger number of response time outliers, the average will decrease dramatically when it is trimmed, while sites with more consistent measurement results will find their average response time is similar with and without the trimmed data.

It is also critical to examine the application’s response times and success rates throughout defined business cycles. A single response time or success rate value eliminates

• variations by time of day
• variations by day of week
• variations by month
• variations caused by advertising and marketing

An average is just an average. If at peak buiness hours, response times are 5.0 seconds slower than the average, then the average is meaningless, as business is being lost to poor performance which has been lost in the focus on the single metric.

All of these items have also fallen prey to their own curse of the single metric. All of the items discussed above aggregate the response time of the business process into a single metric. The process of purchasing items online is broken down into discrete steps, and different parts of this process likely take longer than others. And one step beyond the discrete steps are the objects and data that appear to the customer during these steps.

It is critical to isolate the performance for each step of the process to find the bottlenecks to performance. Then the components in those steps that cause the greatest response time or success rate degradations must be identified and targeted for performance improvement initiatives. If there are one or two poorly performing steps in a business process, focusing performance improvement efforts on these is critical, otherwise precious resources are being wasted in trying to fix parts of the application that are working well.

In summary, a single metric provides a sense of false confidence, the sense that the application can be counted on to deliver response times and success rates that are nearly the same as those simple, single metrics.

The average provides a middle ground, a line that says that is the approximate mid-point of the measurement population. There are measurements above and below this average, and you have to plan around the peaks and valleys, not the open plains. It is critical never to fall victim to the attractive charms that come with the curse of the single metric.

I have set up measurements to monitor the main pages of some of the world’s largest mobile phone providers.

• US Providers
• Canadian Providers
• European Providers
• AsiaPac / South Asia Providers

Just something to do on a rainy Sunday.

Tags: GrabPERF, Web performance, GSM, CDMA, Web site, mobile, wireless, US, Asia, Europe

I have a shiny new Canadian passport that I want to fill with stamps.

Anyone in London need a Web performance analysis from an expert in the field? A detailed examination of your site and a comparison with other national, regional and internationl competitors, complete with a rockin’ statistical breakdown of key metrics?

Daddy needs some airline miles…

Technorati Tags: Passport, Canada, travel, business travel, London

This afternoon, StatCounter showed a marked increase in performance.

Normally I wouldn’t highlight an issue that only lasted an hour, but this appears to have been a very unusual issue that saw the page size decrease to nearly nothing, and performance shoot up to around 45 seconds. This combination usually indicates a back-end application timeout which then presents users with an error message.

StatCounter is in the GrabPERF Site Statistics Index.

Technorati Tags: StatCounter, Web Performance, GrabPERF, Web Metrics, Web Analytics

The Site Statistics | Web Analytics Index measurements have been running now for about 2.5 days, and I wanted to make some general comments on what I am seeing.

The methodolgy for testing is straightforward. I chose sites | services that allowed you to create a free (if limited) account to track your Web visitors, and allowed you to make these statistics available to for anyone to look at. Using this this, a measurement was established against the landing page that visitors would see if they chose to look at these publicly available statistics.

I am using this blog as the placeholder for the tracking “bugs”  used in this index (see the right-hand column).

From the graph above, it is clear that ShinyStat is the performance leader in this space. They have the smallest overall page size as well as the fastest and most reliable performance.

It is important to note that services such as WebTrends, Omniture, WebSideStory and Coremetrics are not included, as they are beyond the reach of most bloggers, and do not provide a public side to their data. Also, Google Analytics is not included, as they do not provide public access to the collected data.

The collected data is available in GrabPERF as both the Site Statistics Index, and as individual measurements.

Technorati Tags: Web analytics, web metrics, shinystat, onestat, statcounter, sitemeter, GrabPERF, Web Performance

James Governor hits for six with this gem.

By bringing their correlation capabilities with Web metrics (Urchin) and site visits (Web Accelerator and Toolbar), Google will be able to direct even better, more focused ad placement, based on visitor location, time of day, Originating ISP, “actual” bandwidth, and any number of other metrics that they will have at their disposal.

I am interested, and concerned, with the brilliance of the Google strategy.

I submitted a presentation proposal for OSCON 2005 just now. The abstract is below.

The Open Source community has driven the online world for the last decade. PHP, PERL, Apache, Java, and MySQL are all major components of large online enterprises.

However, putting an application online and ensuring that it satisfies the performance, availability and reliability demands of the increasingly knowledgeable online consumer are often two separate concerns.

Performance should not be an afterthought; performance should be a leading force in creating a Web application.

Using simple Open Source Tools, Web performance measurement solutions can be built that rival commercial solutions. But what does this data tell you? And how do you turn this into useful business information?

This discussion will expose the participants to key Web performance metrics that make sense to both technology and business leaders in your organization.

I have a snowball’s chance in hell of having it accepted, as it is not hip, technical or trendy, and I am not an Open Source Guru, but if you design stuff for the Web, then you better be ready to have your site examined in detail, because if you don’t do it, your customers will.

In the world of Web performance, the agreed upon state of Nirvana is the development of an automated system that will isolate, identify, diagnose and resolve (or suggest a resolution) to an issue.  However, the question for me is whether these systems are really useful.

Why do I say that? Because they solve the tactical issues. The day-to-day issues. But there is no solution for poor design, inadequate eqipment, overloaded systems, and other strategic decisions. Automated performance systems do not solve the underlying problem — delivering reliable and relevant information on Web performance metrics that matter to business customers.

Who consumes Web performance data? Technology teams.

Who needs holistic Web performance information? Business leaders.

Who does the Web performance industry currently serve?

A few posts ago, I made some statements which I may come to regret. However, as I wrote to the one person who commented on my statement, what I said was more of a back-handed challenge to Canadian employers to show me that they are truly innovative and world-busting.

I issue a challenge to Canadian Internet firms: Show me that you understand Web performance excellence, and are willing to take on a process to implement this concept from the CEO to the receptionist. I want to participate in this; I want to make a Canadian firm the example that the can be shown to the world as the leader.

It may be a niche area, but if you think about it, it means that a company has to understand how its Web property fits into or defines its business model from top to bottom. There are few companies in the world who can say that they understand this, especially not those who did not begin with an explicit ebusiness focus.

There may be firms that think that they have a handle in this. They key question is this: Do your business performance metrics talk to your Web performance metrics?

Let’s work together to make your business and technology speak the same language.