Why GrabPERF?

About four years ago, I had a bright idea that I would like to learn more about how to build and scale a small Web performance measurement platform. I’ve worked in the Web performance industry for nearly a decade now, and this was an experimental platform for me to examine and encounter many of the challenges that I see on a daily basis.

The effort was so successful and garnered enough attention during the initial blogging boom that I was able to sell the whole platform for a tiny (that is not a typo) sum to Technorati.

The name is taken from another experimental tool I wrote called GrabIT2 which uses the PHP cURL libraries to capture timings and HTML data for HTTP requests. It is an extension of my articles and writings on Web performance that started at Webperformance.org, and that have since moved to this blog.

What is GrabERF?

GrabPERF is a multi-location measurement platform, based on PERL, cURL, PHP, and MySQL that is designed to

• Measure the base HTML or a single-object target using HTTP or HTTPS
• Report the data to a central database (located in the San Francisco Area)
• Report the data using a GUI or through text based download

Why not Full Pages with all Objects?

Reason 1: I work for a company that already does that. Lawyers and MBAs among you, do the math.

Reason 2: I am an analyst, not a programmer. The best I can say about my measurement script is hack job.

Why is the GrabPERF interface so clunky?

See reason 2 above.

If you want to write your own interface to the data, let me know.

Why has the interface not changed in nearly three years?

The current interface works. It’s simple, clean, and delivers the data that I and the regular users need to analyze performance issues. If there is something more that you would like to see, let me know!

I like what I see. How can I host a measurement location?

Just contact me, and I can provide you with a list of PERL modules you will need to install on your linux server. In return, I need a static IP address of the machine hosting the measurement agent.

How stable is GrabPERF?

Most of the time, I forget it’s even running. I have logged onto the servers and typed in uptime and discovered that it’s been 6 months or more since the servers have been re-booted.

It was designed to be simple, because that’s all I know how to do. The lack of complexity makes it effectively self-managing.

Shouldn’t all systems be that way?

What if my question isn’t asked / answered here?

Your should know the answer to this by now: contact me.

Tags: experimental tool, GrabIT2, GrabPERF, GrabPERF.org, GUI, HTML, HTTP, Linux, Linux: Server, Perl, PHP, Web Performance, Web performance industry, Web performance measurement platform

The GrabPERF Black Friday Dashboard is done for another year and there were two performance victims that suffered the most at the hands of the onslaught of bargain-hunters in the area of Web performance.

Some caveats that I need to mention about the GrabPERF measurement methodology.

1. Only the base HTML file of each site is measured.
2. Only the base HTML of the homepage is measured. This means that any issues that arose in the shopping process were not captured.

All of the sites in the GrabPERF Holiday Retail Measurement Index can be continually monitored on the GrabPERF Black Friday Dashboard. This page will be available until January 1 2009.

That said, the two primary performance victims this year are HP Shopping and Sears. We focus here on those that did not do that well because sites who have met the Web performance challenge and survived to fight another year are not as interesting from a learning perspective.

HP Shopping

HP Suffered the greatest response time problems, by effectively becoming unresponsive as of 09:00 EST. The greates affect on overall response time came as a result of the First Byte time metric which is a solid proxy for measuring the server or application load, as it is the time between the initial client HTTP request and the server’s HTTP response.

Factored into the poor performance analysis is the fact that GrabPERF only captures data for the base HTML object. If the performance seen here is carried over to the download of all of the graphical content on the page, I would be surprised if anyone was able to make any kind of purchases on the HP web site on Black Friday.

Today, performance has returned to substantially lower levels, indicating that this application was simply not ready for the amount of traffic it received, or ran into a completely unexpected issue when the load increased.

Recommendation for 2009: Load Test the application using this year’s traffic metrics as a baseline for validating the scalability of the application.

Sears

Sears is a returning visitor from last year’s Black Friday measurements. Unfortunately, they return for exactly the same reason that they were on last year - scaling/capacity issues that appear as errors.

And these are the worst kind of errors. As can be seen in the graphic below, the Sears Web site announced to the whole world that they had over-reached and that they could not handle the incoming volume of traffic.

What is interesting is that Sears owns properties that survived the day very well, namely Lands End. The question that must be posed is why does the parent site fail so badly when the child sites handle the traffic without difficulty?

Recommendation for 2009: Load testing for capacity, and meeting with the Lands End team to understand what they are doing to handle the load.

Tags: Black Friday, Black Friday 2008, GrabPERF, HP Shopping, HTML, HTTP, Lands End, Sears, Web Performance, Web performance challenge, Web performance measurement

Once again, the Grabperf Black Friday Dashboard is up and running!

GrabPERF Black Friday Online Retailer Performance Dashboard

Check out your favorite retailer or suggest one we missed!

Tags: Black Friday, Black Friday 2008, GrabPERF, Online Retailer Performance Dashboard, Retailers, Web Performance

I fired up the Boston FIoS measurement location today after a couple of days off, and found that suddenly FIoS doesn’t like the BitTorrent.

The line of purple dots all indicate measurements that reported an error code. All of those measurements come from Boston FiOS. See the real-time graph here.

Accident? Design? That I cannot comment on. I simply report on what I see.

Tags: bittorrent, FIoS, GrabPERF, Verizon, Verizon FIoS, Web Performance, Web performance measurement

In the last 24 hours, thanks to the help of some willing volunteers, GrabPERF has seen the addition of three new measurement locations:

• Dallas, TX (USA)
• Virginia (USA)
• London, UK

All of these location have been graciously provided by the team at e-planning.

Thanks to all of you who volunteer your machines and bandwidth for this project.

As always, we are looking for as more measurement locations. It would be great if we could get some data from the Asia-Pacific region.

Tags: Asia-Pacific, Dallas, e-planning, e-planning.net, External Web Performance, GrabPERF, London, United Kingdom, United States, Virginia, Web Performance, Web performance measurement, West Virginia

Today, there was a network outage that affected the servers from September 21 2008 15:30 GMT until September 22 2008 01:45 GMT.

The data from this period has been cut and hourly averages have been re-calculated.

We apologize for the inconvenience.

Tags: GrabPERF, outage

In 2004-2005, as a lark, I created my own Web performance measurement system, using PERL, PHP and MySQL. In August 2005, I managed to figure out how to include remote agents.

I dubbed it…GrabPERF. An odd name, but an amalgamation of “Grab” and “Performance” that made sense to my mind at the time. I also never though that it would go beyond my house, a couple of basement servers, and a cable modem.

In the intervening three years, I have managed to:

• scale the system to handle over 250 individual measurements
• involve nine remote measurement locations
• move the system to the Technorati datacenter
• provide key operational measurement data to system visitors

Although the system lives in the Technorati datacenter and is owned by them, I provide the majority of the day-to-day maintenance on a volunteer basis, if only to try and keep my limited coding skills up.

But this post is not about me. It’s about GrabPERF.

Thanks to the help of a number of volunteers, I have measurement locations in the San Francisco Bay Area, Washington DC, Boston, Portugal, Germany and Argentina.

While this is a good spread, I am still looking to gather volunteers who can host a GrabPERF measurement location. The areas where GrabPERF has the most need are:

• Asia-Pacific
• South Asia (India, Pakistan, Bangladesh)
• UK and Continental Europe
• Central Europe, including the ancestral homeland of Polska

It would also be great to get a funky logo for the system, so if you are a graphic designer and want to create a cool GrabPERF logo, let me know.

The current measurement system requires Linux, cURL and a few add-on Perl modules. I am sure that I could work on other operating systems, I just haven’t had the opportunity to experiment.

If you or your organization can help, please contact me using the GrabPERF contact form.

Tags: Argentina, Asia-Pacific, Bangladesh, Boston, cable modem, Central Europe, Germany, GrabPERF, GrabPERF.org, India, internet performance measurement data, Linux, mysql, operating systems, Pakistan, performance measurement, Perl, PHP, Polska, Portugal, San Francisco, United Kingdom, Washington DC, web, Web Performance, Web performance measurement data, Web performance measurement system

This is actually a short post to write, as the state of the GrabPERF system is currently very healthy. There was an eight-hour outage in early August 2008, but that was a fiber connectivity issue, not a system issue.

Over the history of ther service, we have been steadily increasing the number of measurements we take each day.

The large leap occurred when a very large number of tests were added to the system on a single day. But based on this data, the system is gathering more than 900,000 measurements every day.

Thanks to all of the people who volunteer their machines and bandwidths to support this effort!

Tags: GrabPERF

It’s the first day back after the last long weekend of the summer. There is a a great amount of news flooding the intertubes, and what happens?

TechCrunch has a small issue.

It’s likely they’ll be back soon, but it’s still an interesting thing to see.

Update - 09:17 EDT (13:17 GMT)

TechCrunch is back up as of 08:49 EDT (12:49 GMT).

Tags: GrabPERF, outage, techcrunch, Web Performance

Between August 08 2008 18:00GMT and August 09 2008 03:00GMT, GrabPERF had a network-related outage at the hosting facility.

Many thanks to the Technorati team for working hard to resolve this complex issue, which eventually turned out to be a loose fiber-cable.

Data from the time period has been cut, so no one’s stats should be affected.

I apologize for the incovenience.

Tags: GrabPERF, outage, Technorati