The GrabPERF database server failed sometime early this morning. The hosting facility is working to install a new machine, and then will begin the long process of restoring from backups and memory.

Updates will be posted here.

UPDATE – Sep 4 2009 22:00 GMT: The database listener is up and data is flowing into the database and can be viewed in the GrabPERF interface. However, I have lost all of the management scripts that aggregate and drop data. These will be critical as the new database server has a substantially smaller drive. There is a larger attached drive, and I will try and mount the data there.

It will likely take more time than I have at the moment to maintain and restore GrabPERF to its pre-existing state. You can expect serious outages and changes to the system in the next few weeks.

[Whining removed. Self-inflicted injuries are always the hardest to bear.]

UPDATE – Sep 5 2009 03:30 GMT: The Database is back up, and absorbing data. Attempts to move it to the larger drive on the system failed, so the entire database is running on an 11GB partition. <GULP>.

The two most vital maintenance scripts are also running the way they should be. I had to rewrite those from very old archives.

Status: Good, but not where I would like it. I will work with Technorati to see if there is something that I’m missing in trying to use the larger partition. Likely it comes down to my own lame-o linux admin skillz.

I want to thank the ops team from Technorati for spending time on this today. They did an amazing job of finding a machine for this database to live on in record time.

I have also learned the hard lesson of backups. May I not have to learn it again.

UPDATE – Sep 5 2009 04:00 GMT: Thanks again to Jerry Huff at Technorati. He pointed out that if I use a symbolic link, I can move the db files over to the large partition with no problem. Storage is no longer an issue.

[And, why you ask, is Tara Hunt (@missrogue) on this post. Hey, when I asked Tagaroo for Technorati images, this is what it gave me. It was a bit of a shock after 8 hours of mind-stretching recovery work, but hey, ask and ye shall receive.]

UPDATE – Sep 7 2009 01:00 GMT: Seems that I got myself into trouble by using the default MySQL configuration that came with the CentOS distro. As a result, I ran out of database connections! Something that I have chided others for, I did myself.

The symptom appeared when I reactivated my logging database, which runs against the same MySQL installation, just in a separate database. It started to use up the default pool of connections (100) and the agents couldn’t report in.

This has been resolved and everything is back to normal.

There will be GrabPERF downtime on May 29-30 2009 as Technorati completes its colocation relocation process. [here] There is no set time for the outage or its length, as the machines have to be physically relocated to the new datacenter, and then integrated into the new network configuration.

I will be contacting the agent locations in the next few hours to update them on this maintenance.

UPDATE: GrabPERF returned to service as of 23:40 GMT on May 30 2009. Thank you for your patience.

About four years ago, I published/posted a number of indices related to GrabPERF data. This is a re-launch of that, starting with the GrabPERF Search Performance Index.

The methodology of the Search Performance Index is straightforward: A number of key search providers are selected and HTTP GET requests are sent that directly pull the results page of a query page that is searching for ‘New York’.

This is a simple process and one that is a close approximation of the actual search process that hundreds of millions of people perform every day.

alias                       Response Time  Success  Attempts  Success Rate
--------------------------  -------------  -------  --------  ------------
SEARCH - Google Blogsearch      0.2822545    27242     27244       99.9927
SEARCH - Google                 0.3151932    27228     27247       99.9303
SEARCH - Live (Microsoft)       0.3840097    27245     27246       99.9963
SEARCH - Indeed                 0.4112960    27240     27241       99.9963
SEARCH - Yahoo                  0.4574381    24175     24175      100.0000
SEARCH - Altavista              0.4592764    23922     23922      100.0000
SEARCH - Cuil                   0.6757475    23963     23967       99.9833
SEARCH - AOL                    0.7822945    23913     23913      100.0000
SEARCH - Ask                    0.9025220    24157     24163       99.9752
SEARCH - Technorati             0.9053472    27219     27234       99.9449
SEARCH - Amazon                 1.3251402    27245     27251       99.9780
SEARCH - Baidu                  1.7409345    23777     23799       99.9076
SEARCH - Blogdigger             1.8960633    25106     26354       95.2645
SEARCH - BlogLines              2.0238809    27233     27248       99.9450
SEARCH - IceRocket              2.1233684    24147     24177       99.8759
SEARCH - Blogpulse              2.4144131    27019     27247       99.1632

As can be seen in the data, there is a substantial degree of difference in the response time results, but the big three search providers (Google, Yahoo, and Microsoft Live) were in the top five. As well, all but three of the providers (Blogdigger, IceRocket and Blogpulse) had success rates (availability) of 99.9% or higher.

I don’t expect to see much in the way of change in these results, but I will post them for comparison each week.

Today I would like to announce the availability of the GrabPERF Search Performance Index.

The goal of the index is to provide performance metrics for a group of search providers around the world. The results are based on a direct HTTP GET request being made for the search results page by the GrabPERF Agent.

Currently only live data is available. In the near future, archival results on a week by week basis will be made available.

If there is a search provider that has been missed, please contact the GrabPERF team.

Yesterday in the Fast Company Live Fail Whale session [mention on Scoble's blog here], Paul Bucheit of FriendFeed jokingly said that his company’s external alerting mechanism was Louis Gray.

I cringed when I read that, as the last people who should be letting you know you have an issue are your visitors or customers. I know that FriendFeed is new and may not have the ops team that Dorion Carroll and Technorati have developed over the years, but it is still critical.

You have done a lot as a company to build a brand. Don’t let your internal and external performance sully your reputation. There are a number of low-cost and free ways to watch your performance and alert you before things break.

Louis Gray is a great guy. But he is not an objective and reliable way to alert you when something is wrong with your site.

<sarcasm>

Currently, the Newest Industry sits at #1,106,225 in the Technorati Charts. I’m looking to break into the Technorati 1,000,000 before Christmas.

If Chris Brogan can get into the Technorati 100, I know I can do this!

Help me break into this elite group!

</sarcasm>

There is clear dissatisfaction with the current state of marketing among the social media mavens.

• Fred Wilson and Union Square Ventures are looking for companies to invest in to take advantage of this.
• BuzzLogic releases their conversational ad service.
• The Inquisitr moves from AdSense to Technorati Media, indicating a potential shift at b5 Media.
• Lookery is providing demographic information that is relevant to social media providers and shaping ad presentation that way
• A completely off-the-radar person such as myself realizes that there is gold in what holds the attention, and that’s not always the latest and greatest

So what can be done? Jeff Jarvis points out that the problem lies with measurement. I agree, as there is only value in a system where all of the people involved agree on what the metric of record will be, and how it can be validly captured.

Currently CPM is the agreed upon metric. In a feed based online world, how does a CPM model work? And, most importantly, why would I continue to place your ads on my site if all your doing is advertising to people based on the words on the page, rather than who is looking at the page and how often that page is looked at.

In effect, advertisers should be the ones thrying to figure out how to get into the community, get into the conversation. As an advertiser, don’t you want to be where the action is? But how do you find an engaged audience in an online world that makes a sand castle on the beach in a hurricane look stable?

The challenge for advertisers is to be able to find the active communities and conversations effectively. The challenge for content creators and communities is to understand the value of their conversations, the interactions that people who visit the site have with the content.

In effect, a social media advertising model turns the current model on its head. Site owners and community creators gain the benefit of being attractive to advertisers because of the community, not because of the content. And site owners who understand who visits their site, what content most engages them, how they interact with the system will be able to reap the greatest rewards by selling their community as a marketable entity.

And Steven Hodson rounds out the week’s think on communities by throwing out the subversive idea that communities are not always free (as in ‘beer’, not as in ‘land of’). If a community has paid for the privilege of coming together to participate in communal events and discussions, then can’t that become an area for site owners to further control the cost of advertising on their site?

While the benefit of reduced or no marketing content is the benefit of many for-pay communities, this benefit can be used by site owners by saying that an advertiser can have access to the for-pay community at the cost of higher ad rates and smaller ads. The free community is a completely different set of rules, but there are also areas in the free community that are of higher value than others.

In summary, the current model is broken. But there is no way to measure the value of a Twitter stream, a FriendFeed conversation, a Disqus thread, or a Digg rampage. And until there is, we are stuck with an ad model that based on the words on the page, and not the community that created the words.

Between August 08 2008 18:00GMT and August 09 2008 03:00GMT, GrabPERF had a network-related outage at the hosting facility.

Many thanks to the Technorati team for working hard to resolve this complex issue, which eventually turned out to be a loose fiber-cable.

Data from the time period has been cut, so no one’s stats should be affected.

I apologize for the incovenience.

You hear a song off Bob Mould’s album Workbook used in a TIAA-CREF commercial.

I think I’ll just nip off and shoot myself.

Many, many years ago, I started my depressing voyage through the world of work at an Overwaitea food store in my home town. As a teenager, I expected to work so weird hours, and accept some level of abuse from the “adults” I worked with.

However, it seems that the organization now expects all their employees to accept this crap [here].

A major B.C. grocery chain wants some of its unionized staff to work shifts of just two hours, a move the union representing 8,500 workers called shocking.

The Overwaitea Food Group, which also runs Save-On-Foods and Urban Fare, made the demand for two-hour shifts as it began negotiating a new contract with the United Food and Commercial Workers Union, the union said.