Measure what is measurable, and make measurable what is not so.

Galileo Galilei

The greatest challenge facing companies today is not finding ways to measure performance. The key issue is one of understanding what should be measured and validating that there is agreement on what the purpose of the measurement is.

Organizations are complex. And with complexity arises the need to gather data for different purposes. In my series discussing Why Web Measurements?, I broke organizations down into four groups, each one having distinctly different needs for measurements and data. While this series focuses on Web performance, the four categories (Customer Generation, Customer Retention, Business Operations, and Technical Operations) can be broadly applied to all aspects of your business.

In each of the four categories, whether it is for Web performance or financial analysis, determining what and why to measure is a critical predecessor to the establishment of measurements and the examination of data.

2009 will be a year of reflection and retrenchment. Companies will be examining all aspects of their business, all of their relationships with vendors, all of the ways they measure themselves. The question that must be asked before succumbing to the rushing panic of cost-cutting and layoffs is: Do you fundamentally understand why and what you measure and what it is really telling you?

Tags: business operations, Customer Generation, Customer Retention, measurement, Measurement and Methodology, Methodology, performance measurement, Performance Methodology, technical operations, Web Measurements, Web Performance

In 2004-2005, as a lark, I created my own Web performance measurement system, using PERL, PHP and MySQL. In August 2005, I managed to figure out how to include remote agents.

I dubbed it…GrabPERF. An odd name, but an amalgamation of “Grab” and “Performance” that made sense to my mind at the time. I also never though that it would go beyond my house, a couple of basement servers, and a cable modem.

In the intervening three years, I have managed to:

• scale the system to handle over 250 individual measurements
• involve nine remote measurement locations
• move the system to the Technorati datacenter
• provide key operational measurement data to system visitors

Although the system lives in the Technorati datacenter and is owned by them, I provide the majority of the day-to-day maintenance on a volunteer basis, if only to try and keep my limited coding skills up.

But this post is not about me. It’s about GrabPERF.

Thanks to the help of a number of volunteers, I have measurement locations in the San Francisco Bay Area, Washington DC, Boston, Portugal, Germany and Argentina.

While this is a good spread, I am still looking to gather volunteers who can host a GrabPERF measurement location. The areas where GrabPERF has the most need are:

• Asia-Pacific
• South Asia (India, Pakistan, Bangladesh)
• UK and Continental Europe
• Central Europe, including the ancestral homeland of Polska

It would also be great to get a funky logo for the system, so if you are a graphic designer and want to create a cool GrabPERF logo, let me know.

The current measurement system requires Linux, cURL and a few add-on Perl modules. I am sure that I could work on other operating systems, I just haven’t had the opportunity to experiment.

If you or your organization can help, please contact me using the GrabPERF contact form.

Tags: Argentina, Asia-Pacific, Bangladesh, Boston, cable modem, Central Europe, Germany, GrabPERF, GrabPERF.org, India, internet performance measurement data, Linux, mysql, operating systems, Pakistan, performance measurement, Perl, PHP, Polska, Portugal, San Francisco, United Kingdom, Washington DC, web, Web Performance, Web performance measurement data, Web performance measurement system

In the GigaOm blog today, Allen Leinwand puts up a monstrous wake-up call to all the hip and cool Web 2.0 companies out there: Your apps run across the Internet [here].

I have spent 9 years investigating, diagnosing, and validating the Web performance issues of companies. I can tear the Web performance data of a site down quickly and ask pointed questions about why certain components of an application are behaving poorly.

But even after 9 years, there are still gimme problems around connection setup that I can seem brilliant about, not because I have some secret knowledge, but because I think of Web performance from the Network UP, not from the Application DOWN.

The subtlety of this difference what Leinwand is alluding to. Fancy applications run across the Internet. The Internet is built on TCP. And TCP is built on-top of a very complex networking infrastructure that is way beyond the realm of my skills.

If you don’t know what packet loss looks like, or how your fancy app presents to clients, or how to ensure that this data is collected and presented to you in a timely way, then you are being exposed to alerting by client calls.

All because you thought the biggest problem was scaling your app, not ensuring that the network it crossed to reach people affected the way it performed. Network geeks created Web 1.0; Web 2.0 seems to think they are mostly unecessary.

Wrong.

Measure your performance. Understand TCP. Hire a network geek (or 20).

Then sleep better at night.

Tags: Allen Leinwand, GigaOm, Internet, networking, performance measurement, TCP, Web 1.0

mon.itor.us, a service which also provides free Web performance measurement services, appears to be having a wee problem.

The most recent GrabPERF data on this site is available here. The issue may be corrected by the time you look at the data.

I don’t wish suffering like this on anyone. GrabPERF had it’s own 3-4 day outage a few months ago. It’s just sad to see when monitoring services go down.

Tags: app, ARIN, data, GrabPERF, GrabPERF.org, HTTP, IE, IM, it, measurement, Om, One, outage, performance, performance measurement, PHP, pr, web, Web Performance, Web performance measurement

This morning, after months of increasing performance issues, and connectivity issues, I have retired the Boston Verizon measurement location. This location hosted 2 measurement agents.

The machines, hosted in my basement, are connected using Verizon FiOS, which has become increasingly flaky over the last couple of months. As well, the machines are 7 year old Pentium IIs, and require a substantial amount of spoon-feeding that I don’t have time for at the moment.

I have re-enabled the Technorati #1 Agent to fill the gap, but this leaves only 4 measurement locations running. I again put out my plea for more volunteers to host GrabPERF measurement locations. I have had one volunteer contact me about this (thanks Henrik!), and this location should be up in about 3 weeks.

If you have a spare linux box and a static IP, have I got a project for you!

PS: The contact page is fixed. It was set to auto-refresh and overwrite your e-mails. Ooooops.

Tags: GrabPERF, Agent, Verizon, Web Performance, measurement, performance measurement

Tags: Boston, connectivity, GrabPERF, GrabPERF.org, HTTP, IM, IP, Issues, it, leave, Linux, Mac, measurement, Om, One, performance, performance issues, performance measurement, PHP, pr, run, Technorati, tv, Verizon, web, Web Performance, Web+performance

The weekly GrabPERF Search Index Results are in.

NOTES:

• Level3/Cogent Peering Dispute affected all results this week

Week of October 04-10, 2005

TEST                                 RESULT  SUCCESS  ATTEMPTS
--------------------------------  ---------  -------  --------
PubSub - Search                   0.4032768    99.86      6644
Google - Search                   0.5196460   100.00      6735
Yahoo - Search                    0.6926705   100.00      6694
MSN - Search                      0.7543688   100.00      6722
Google Blogsearch (ATOM)- Search  0.8480318    99.82      6725
Google Blogsearch - Search        0.8880010    99.84      6731
eBay - Search                     0.9723281   100.00      6736
Newsgator - Search                0.9935382    99.61      6685
BestBuy.com - Search              1.2988528    99.94      6721
Findory - Search                  1.3819290    99.93      6693
Technorati - Search               1.4609834    99.88      6671
Amazon - Search                   1.6579611    98.94      1799
BENCHMARK RESULTS                 1.7649221    99.56    108774
Feedster - Search                 2.4226014    99.73      6671
IceRocket - Search                2.6644861    99.51      6698
Blogdigger - Search               2.7916678    99.14      6378
BlogLines - Search                3.5648172    96.84      6735
Blogpulse - Search                6.6344053    98.96      6736

These results are based on data gathered from two remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.

The measurements are for the base HTML document only. No images or referenced files are included.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

Tags: app, blog, data, digg, feeds, Google, GrabPERF, GrabPERF.org, HTML, HTTP, IM, ISP, it, measurement, North America, Om, performance, performance measurement, pr, search, seo, Technorati, web, Web Performance, Web performance measurement, Web+performance

The weekly GrabPERF Search Index Results are in.

NOTES:

• Findory blog search added to index
• Google standard search takes top spot with 100% availability

Week of September 26 - October 03, 2005

TEST                                 RESULT  SUCCESS  ATTEMPTS
--------------------------------  ---------  -------  --------
Google - Search                   0.2477162   100.00      7062
PubSub - Search                   0.2538995    99.80      7043
Google Blogsearch (ATOM)- Search  0.5839898    99.90      7064
MSN - Search                      0.6146201    99.99      7063
Yahoo - Search                    0.6154676    99.99      7060
Google Blogsearch - Search        0.6547768    99.92      7064
eBay - Search                     0.6940026   100.00      7058
BestBuy.com - Search              1.0745931    99.91      7056
Technorati - Search               1.1970990    99.92      7062
Findory - Search                  1.2055845    99.91      6725
Amazon - Search                   1.2378869    99.33      1803
Feedster - Search                 1.2583286    99.92      7061
BENCHMARK RESULTS                 1.4132893    99.73    114382
Newsgator - Search                1.5429696    99.29      7062
Blogdigger - Search               1.7054253    99.99      7058
BlogLines - Search                1.7401694    98.99      7061
IceRocket - Search                3.7427026    98.57      7012
Blogpulse - Search                5.5294509    99.72      7062

These results are based on data gathered from two remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.

The measurements are for the base HTML document only. No images or referenced files are included.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

Tags: app, blog, data, digg, feeds, Google, GrabPERF, GrabPERF.org, HTML, HTTP, IM, it, measurement, North America, Om, performance, performance measurement, pr, search, seo, Technorati, web, Web Performance, Web performance measurement, Web+performance

The weekly GrabPERF Search Index Results are in.

Week of September 19-25, 2005

TEST                                 RESULT  SUCCESS  ATTEMPTS
--------------------------------  ---------  -------  --------
PubSub - Search                   0.1641161    99.82      7037
Google - Search                   0.2469639   100.00      7037
Google Blogsearch (ATOM)- Search  0.5754907    99.89      6201
Google Blogsearch - Search        0.5844869    99.67      7037
MSN - Search                      0.6129932    99.99      7036
Yahoo - Search                    0.6375895    99.94      7031
eBay - Search                     0.6750205   100.00      7036
Amazon - Search                   1.0858985    99.95      1832
Technorati - Search               1.1052304    99.89      7036
BestBuy.com - Search              1.3558370    99.52      7036
Newsgator - Search                1.5025822    99.97      7037
Feedster - Search                 1.5255779    99.52      7036
BENCHMARK RESULTS                 1.5572177    99.66    106498
Blogdigger - Search               1.7119017    99.94      7016
BlogLines - Search                2.6147261    97.24      7036
IceRocket - Search                3.4088173    99.86      7018
Blogpulse - Search                6.6481991    99.55      7036

These results are based on data gathered from two remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.

The measurements are for the base HTML document only. No images or referenced files are included.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

Tags: app, blog, data, digg, feeds, Google, GrabPERF, GrabPERF.org, HTML, HTTP, IM, it, measurement, North America, Om, performance, performance measurement, pr, search, seo, Technorati, web, Web Performance, Web performance measurement, Web+performance

The weekly GrabPERF Search Index Results are in.

This Week’s Notes

1. ERTW.com Agent location taken offline
2. Addition of Google Blogsearch to the Index
3. Numerous performance improvements to the GrabPERF interface

Week of September 12-18, 2005

TEST                           RESULT  SUCCESS  ATTEMPTS
--------------------------  ---------  -------  --------
PubSub - Search             0.2688096    99.82      6532
Google - Search             0.4013164    99.97      6532
Google Blogsearch - Search  0.5818507    98.60      4214
MSN - Search                0.6981630    99.83      6532
Yahoo - Search              0.7159974    99.95      6527
eBay - Search               0.8345692   100.00      6528
BlogLines - Search          1.0204595    99.95      6531
BestBuy.com - Search        1.1687228    99.97      6530
Feedster - Search           1.3112797    99.82      6531
Technorati - Search         1.3240335    99.95      6528
Amazon - Search             1.5195445    99.72      2481
Newsgator - Search          1.5823492    99.72      6529
Blogdigger - Search         1.7142475    99.97      6506
BENCHMARK RESULTS           2.0313721    99.50     76849
IceRocket - Search          4.2792600    98.79      6515
Blogpulse - Search          6.5226776    99.29      6522

These results are based on data gathered from two remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.

The measurements are for the base HTML document only. No images or referenced files are included.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

Tags: app, blog, data, digg, feeds, Google, GrabPERF, GrabPERF.org, HTML, HTTP, IM, improvement, it, measurement, North America, Om, performance, performance improvement, performance measurement, pr, search, seo, Technorati, web, Web Performance, Web performance measurement, Web+performance

The weekly GrabPERF Search Index Results are in.

Week of September 4-11, 2005

TEST                     RESULT  SUCCESS  ATTEMPTS
--------------------  ---------  -------  --------
PubSub - Search       0.5421396    99.72      5680
Google - Search       0.5741554   100.00      5681
Yahoo - Search        0.8028792    99.95      5678
eBay - Search         0.9255761   100.00      5681
Feedster - Search     1.2524916    99.86      5679
MSN - Search          1.2737601    98.40      5681
BestBuy.com - Search  1.3964269    99.91      5680
Technorati - Search   1.4128937    99.98      5681
BlogLines - Search    1.6020669    99.16      5681
Amazon - Search       1.9909649    99.74      3108
BENCHMARK RESULTS     2.0313721    99.50     76849
Blogdigger - Search   2.3336642    98.53      5577
Newsgator - Search    2.5181818    99.68      5681
IceRocket - Search    4.3043685    99.77      5680
Blogpulse - Search    7.4959768    98.40      5681

These results are based on data gathered from three remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.

The measurements are for the base HTML document only. No images or referenced files are included.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO

Tags: app, blog, data, digg, feeds, Google, GrabPERF, GrabPERF.org, HTML, HTTP, IM, it, measurement, North America, Om, performance, performance measurement, pr, search, seo, Technorati, web, Web Performance, Web performance measurement, Web+performance