Why GrabPERF?

About four years ago, I had a bright idea that I would like to learn more about how to build and scale a small Web performance measurement platform. I’ve worked in the Web performance industry for nearly a decade now, and this was an experimental platform for me to examine and encounter many of the challenges that I see on a daily basis.

The effort was so successful and garnered enough attention during the initial blogging boom that I was able to sell the whole platform for a tiny (that is not a typo) sum to Technorati.

The name is taken from another experimental tool I wrote called GrabIT2 which uses the PHP cURL libraries to capture timings and HTML data for HTTP requests. It is an extension of my articles and writings on Web performance that started at Webperformance.org, and that have since moved to this blog.

What is GrabERF?

GrabPERF is a multi-location measurement platform, based on PERL, cURL, PHP, and MySQL that is designed to

• Measure the base HTML or a single-object target using HTTP or HTTPS
• Report the data to a central database (located in the San Francisco Area)
• Report the data using a GUI or through text based download

Why not Full Pages with all Objects?

Reason 1: I work for a company that already does that. Lawyers and MBAs among you, do the math.

Reason 2: I am an analyst, not a programmer. The best I can say about my measurement script is hack job.

Why is the GrabPERF interface so clunky?

See reason 2 above.

If you want to write your own interface to the data, let me know.

Why has the interface not changed in nearly three years?

The current interface works. It’s simple, clean, and delivers the data that I and the regular users need to analyze performance issues. If there is something more that you would like to see, let me know!

I like what I see. How can I host a measurement location?

Just contact me, and I can provide you with a list of PERL modules you will need to install on your linux server. In return, I need a static IP address of the machine hosting the measurement agent.

How stable is GrabPERF?

Most of the time, I forget it’s even running. I have logged onto the servers and typed in uptime and discovered that it’s been 6 months or more since the servers have been re-booted.

It was designed to be simple, because that’s all I know how to do. The lack of complexity makes it effectively self-managing.

Shouldn’t all systems be that way?

What if my question isn’t asked / answered here?

Your should know the answer to this by now: contact me.

Tags: experimental tool, GrabIT2, GrabPERF, GrabPERF.org, GUI, HTML, HTTP, Linux, Linux: Server, Perl, PHP, Web Performance, Web performance industry, Web performance measurement platform

The GrabPERF Black Friday Dashboard is done for another year and there were two performance victims that suffered the most at the hands of the onslaught of bargain-hunters in the area of Web performance.

Some caveats that I need to mention about the GrabPERF measurement methodology.

1. Only the base HTML file of each site is measured.
2. Only the base HTML of the homepage is measured. This means that any issues that arose in the shopping process were not captured.

All of the sites in the GrabPERF Holiday Retail Measurement Index can be continually monitored on the GrabPERF Black Friday Dashboard. This page will be available until January 1 2009.

That said, the two primary performance victims this year are HP Shopping and Sears. We focus here on those that did not do that well because sites who have met the Web performance challenge and survived to fight another year are not as interesting from a learning perspective.

HP Shopping

HP Suffered the greatest response time problems, by effectively becoming unresponsive as of 09:00 EST. The greates affect on overall response time came as a result of the First Byte time metric which is a solid proxy for measuring the server or application load, as it is the time between the initial client HTTP request and the server’s HTTP response.

Factored into the poor performance analysis is the fact that GrabPERF only captures data for the base HTML object. If the performance seen here is carried over to the download of all of the graphical content on the page, I would be surprised if anyone was able to make any kind of purchases on the HP web site on Black Friday.

Today, performance has returned to substantially lower levels, indicating that this application was simply not ready for the amount of traffic it received, or ran into a completely unexpected issue when the load increased.

Recommendation for 2009: Load Test the application using this year’s traffic metrics as a baseline for validating the scalability of the application.

Sears

Sears is a returning visitor from last year’s Black Friday measurements. Unfortunately, they return for exactly the same reason that they were on last year - scaling/capacity issues that appear as errors.

And these are the worst kind of errors. As can be seen in the graphic below, the Sears Web site announced to the whole world that they had over-reached and that they could not handle the incoming volume of traffic.

What is interesting is that Sears owns properties that survived the day very well, namely Lands End. The question that must be posed is why does the parent site fail so badly when the child sites handle the traffic without difficulty?

Recommendation for 2009: Load testing for capacity, and meeting with the Lands End team to understand what they are doing to handle the load.

Tags: Black Friday, Black Friday 2008, GrabPERF, HP Shopping, HTML, HTTP, Lands End, Sears, Web Performance, Web performance challenge, Web performance measurement

There is clear dissatisfaction with the current state of marketing among the social media mavens.

• Fred Wilson and Union Square Ventures are looking for companies to invest in to take advantage of this.
• BuzzLogic releases their conversational ad service.
• The Inquisitr moves from AdSense to Technorati Media, indicating a potential shift at b5 Media.
• Lookery is providing demographic information that is relevant to social media providers and shaping ad presentation that way
• A completely off-the-radar person such as myself realizes that there is gold in what holds the attention, and that’s not always the latest and greatest

So what can be done? Jeff Jarvis points out that the problem lies with measurement. I agree, as there is only value in a system where all of the people involved agree on what the metric of record will be, and how it can be validly captured.

Currently CPM is the agreed upon metric. In a feed based online world, how does a CPM model work? And, most importantly, why would I continue to place your ads on my site if all your doing is advertising to people based on the words on the page, rather than who is looking at the page and how often that page is looked at.

In effect, advertisers should be the ones thrying to figure out how to get into the community, get into the conversation. As an advertiser, don’t you want to be where the action is? But how do you find an engaged audience in an online world that makes a sand castle on the beach in a hurricane look stable?

The challenge for advertisers is to be able to find the active communities and conversations effectively. The challenge for content creators and communities is to understand the value of their conversations, the interactions that people who visit the site have with the content.

In effect, a social media advertising model turns the current model on its head. Site owners and community creators gain the benefit of being attractive to advertisers because of the community, not because of the content. And site owners who understand who visits their site, what content most engages them, how they interact with the system will be able to reap the greatest rewards by selling their community as a marketable entity.

And Steven Hodson rounds out the week’s think on communities by throwing out the subversive idea that communities are not always free (as in ‘beer’, not as in ‘land of’). If a community has paid for the privilege of coming together to participate in communal events and discussions, then can’t that become an area for site owners to further control the cost of advertising on their site?

While the benefit of reduced or no marketing content is the benefit of many for-pay communities, this benefit can be used by site owners by saying that an advertiser can have access to the for-pay community at the cost of higher ad rates and smaller ads. The free community is a completely different set of rules, but there are also areas in the free community that are of higher value than others.

In summary, the current model is broken. But there is no way to measure the value of a Twitter stream, a FriendFeed conversation, a Disqus thread, or a Digg rampage. And until there is, we are stuck with an ad model that based on the words on the page, and not the community that created the words.

Tags: ad network, ads, adsense, advertising, advertising model, API, attention, audience, b5 Media, blog, blog advertising, BuzzLogic, BuzzMachine, community, content, control, conversation, current, data, digg, disqus, EAD, feeds, Fred Wilson, friendfeed, HTML, HTTP, IE, IM, information, interaction, interactions, IP, it, Jeff Jarvis, LAN, Lookery, Mac, marketing, measurement, media, metrics, Newest Industry, Om, One, online, online world, pr, social media, social media advertising, social media advertising model turns, social media mavens, social media providers, Steven Hodson, Technorati, The Inquisitr, twitter, Union Square, Union Square Ventures

The intertubes of the Web are abuzz with talk of the new, open-source Google Chrome browser [two articles here and here]. I will not presume to wade into the debate of whether it is necessary, or what strategic business goals Google has set that rely on having its own browser. I will limit my comments to the area of Web performance.

Open-Source Browser: Ours or Theirs?

When I read that Google Chrome was an open-source browser, the first thought was: is it theirs or a re-branded Firefox? No one knows at this point, but that will have a direct effect on how the browser performs, and how extensible it will be.

HTTP Standards

Unlike other standards, HTTP standards set out how a browser uses the underlying TCP stack. MSIE6/7 have very broken implementations, and MSIE8 is building on those by increasing the number of connections per host to 6, up from 2 set out in RFC 2616.

Firefox can be configured to mangle this as well, but by default it plays by the standard, adding the option of HTTP pipelining into its mix of persistent HTTP connections.

It will be VERY interesting to see how Google Chrome comes configured out of the box, and how much control users have over the HTTP behaviour of this new browser.

(X)HTML/CSS/JS Standards

This area is a mess. No browser implements this standards in a way that is completely consistent with the written text, and page designers have to use a variety of page testing products (such as BrowserCam) prior to release to ensure that their design is somewhat presentable in all browsers on all platforms.

The rendering of Javascript will be crucial in this new browser, as so much of the new Web is built on applications that are almost completely Javascript-driven.

I am sure that there will be sites that will be completely mangled by the new browser, but, knowing Google, we will be getting a 2.0 release, the 1.0 release being used within Google for a while now to test it under real-world conditions.

Caching

As a few sites in the world do use cache-control headers properly, it will be interesting to see how a browser created by one of the major ad-serving and search providers on the Web tracks page objects. Will it follow explicit/implicit caching rules? Or will it impose a heavy penalty on bandwidth by downloading objects more frequently than other production browsers do?

Proxies, and the Debacle of the Google Web Accelerator

Back in 2005, Google launched a badly designed and gighly flawed product called the Google Web Accelerator. This product proxied Web traffic through the Google network and allowed the company to develop a pattern of user browsing habits and search selections that would allow them to better target their ad products.

I have a great fear that this will be an integrated part of the Google browser project. If it is, it should be a configurable option, not an out-of-the box standard.

I am sure that there will be a few performance conversations that occur around the Google Chrome browser in the weeks ahead. I look forward to hearing what the community has to say about this new addition to the browser wars.

Tags: browser, browser war, browser wars, css, Firefox, Google Web Accelerator, HTML, HTTP, HTTP pipelining, js, MSIE, persistent connections, proxy, XHTML

Many, many years ago, I started my depressing voyage through the world of work at an Overwaitea food store in my home town. As a teenager, I expected to work so weird hours, and accept some level of abuse from the “adults” I worked with.

However, it seems that the organization now expects all their employees to accept this crap [here].

A major B.C. grocery chain wants some of its unionized staff to work shifts of just two hours, a move the union representing 8,500 workers called shocking.

The Overwaitea Food Group, which also runs Save-On-Foods and Urban Fare, made the demand for two-hour shifts as it began negotiating a new contract with the United Food and Commercial Workers Union, the union said.

Tags: Canada, cbc, content, HTML, HTTP, ISP, it, Om, pr, run, Technorati, teen, teenager

It has been at least a year since I last updated everyone on the state of GrabPERF. That’s because for most of the last year, the system has been rolling along without a hitch or a major systemic change. The last major change to the agent code was alomost exactly a year ago, when I added the ability to capture text matches.

It wasn’t until last week, however, that I allowed folks to be able to see the results of these text match failures. Let’s just say that motivation has been low and my real job has been keeping me busy.

I did want to share the growth, and mellowing of the system as it progresses into year three.

Total Measurements Per Day

Unique Tests Measured Per Day

Back in July 2005 when I started this grand experiment, I was gathering 10,000 measurements a day from less than forty tests. The system spiked at 390,000 measurements per day (April 2006) and 147 tests (June 2006).

Starting just after that, I started reducing the number of tests to improve system efficiency, and began developing the text match capability.

There have been some changes to the number of measurements, but on the whole, the system has been completely stable for the last 12 months.

As some of you may have noted, I have add some new features in the last 10 days, and re-organized the structure of the system to allow for better tracking of usage. Over the next few months, I will be attacking the code to make it process things more efficiently, but not substantially change the appearance or functionality of GrabPERF.

It has been noted by some commentators that the design doesn’t pop and sizzle. No AJAX, DHTML, or other flashy gizmos. Guess what? The system is designed to deliver data efficiently and effectively. And as someone who has seen the performance fall-out from badly delivered Web 2.0 implementations, I will stick with clunky and effective, as, in the end, you gotta put all those bytes on the wire.

For those that have stuck with the system over the last year, thanks. I enjoy delivering the best measurement data money can’t buy, and hope you stick around for the ride.

Tags: ajax, app, ARIN, cbc, content, data, design, GrabPERF, HTML, HTTP, IE, IM, ISP, it, live, measurement, Om, One, performance, pr, Technorati, tracking, update, web, web 2.0, Web Performance

A colleague of mine suggested that the Tour de France give up all pretense of being drug free, and embrace the performance-enhancing image it has developed so carefully over the last quarter century.

His idea was to have the racing teams sponsored by the major pharmaceutical firms, pitting one performance-enhancing approach against another, in a competition to demonstrate not the strength of the human spirit, but the power to manipulate the human body.

Samantha further suggested that they then abandon all of the rules that make the race civilized, and turn it into a free-for-all, a Tour de France, EXTREME EDITION!

That would get the le tour on Spike.

Tags: app, content, France, HTML, HTTP, IM, IP, ISP, it, Mac, Om, One, performance, pr, Technorati, tv

It seems that more than the usual immigration backlog reduction process has been at work in the USCIS. There are two likely scenarios that appear to be running around immigration circles these days, regarding the Green Card slot tease that has turned into such a furore.

The first is that the Department of State, which issues the Visas, was pressuring the USCIS to fill the Fiscal 2007 Green Card quota, something that has happened rarely in the last few years. What most people in the US don’t know is that most years, thousands of eligible Green Card slots simply disappear because the applications can’t be processed fast enough by the USCIS.

Recent events have highlighted this, and the Department of State may have applied pressure to USCIS to completely exhaust the 2007 pool, to avoid the embarrassment of having to explain to Congress why they can’t process applications faster.

The second reason is greed: as of August 1 2007, the government fees for Green card applications increases massively. For a family of four, the cost will increase by $2,500. So, by not allowing the flood of applications from all of those expectant people, they have guaranteed themselves a higher revenue stream for next year.

All things considered, the whole event smells.

Now, for the long-term affect on skilled immigrants, Microsoft has set the trend by announcing that it will be moving development over the border to Canada [here]. As a country with a skills-based immigration policy, highly-trained technical professionals feel welcomed and wanted in Canada, something that is not the case with the archaic and glacial immigration policy of the United States.

In the next 5-10 years, US companies will face a serious inability to recruit employees from anywhere other than the United States. Skilled professionals will simply not come to a country that actively discourages them from staying permanently and making a contribution.

The US policy policy will be a boon to Canada, Ireland, and other countries who actively seek and encourage skilled professional immigrants.

Tags: USCIS, Department of State, immigration, Green Card, skilled professionals, Canada, Ireland, brain drain

Tags: app, application, Canada, Challenges, cro, government, Green Card, HTML, HTTP, IE, IM, immigrants, Immigration, Ireland, Issues, it, LAN, Microsoft, Om, pr, run, Technorati, United States

I have been reading Greg Siskind’s blog, and he has many articles on the growing hue and cry over the USCIS Visa debacle. [here and here, as examples]

Based on everything I have read, I might benefit from this scam. However, it makes me ill to think that a group of bureaucrats broke their own rules in order to boost their Visa acceptance rate, which is what it sounds like.

Tags: Greg Siskind, USCIS, Green Card, immigration

Tags: app, blog, EAD, Green Card, HTML, HTTP, IM, Immigration, it, Om, Technorati

As many readers know, I am going through the process — if you call filing a bunch of paperwork and not hearing anything for 2 years a process — of obtaining Permanent Residency in the United States, often referred to as the Green Card.

This morning, on NPR, there was a story about a foul-up in the processing of Green Cards that is suspicious, to say the least.

I have started referring to this process as the Dream Card because it leaves one thinking that the application they completed was done in a dream, a long time ago. An like most dreams, it is a fable of the subconscious mind and as likely to come true as those blue, flying penguins in my dream last night.

The degree of complexity that accompanies the application process has made bureaucrats from the Byzantine Empire write letters of complaint to their members of Congress, saying that the USCIS is giving them a bad name. Kafka has been seen rising from the dead at night, and penning a new tale based on this experience.

Other people covering this story.

NY Times
The Guardian
Times Of India
Miami Herald
San Jose Mercury News
Sacramento Bee Editorial

A few media outlets have grabbed this story as an example of just how broken the US system is when it comes to immigration, especially given the irony of the recent debate over the immigration bill that was tossed out of Congress. How could the immigration system have hoped to deal with the new regulations, if thousands, perhaps tens of thousands, of valid visas go unused every year, due to government inefficiency.

Why would an illegal immigrant bother to go through a legal process that punishes the very people who are taking the time to follow the rules?

I would raise my voice in protest; but it would do no good. Drawing a pool of highly skilled, well compensated indentured servants from around the world to these shores to keep the wheels of innovation and development rolling appears to have become the American way.

And like indentured servants everywhere, we are a disposable commodity, to be teased by the promise that some day, we could, we might, just maybe be able to live here (and still not be able to vote) as Permanent Residents.

Tags: Green Card, fiasco, scandal, State Department, USCIS, U.S. Citizenship and Immigration Services, indentured, servants, work visa, visa

Tags: app, application, ARIN, dreams, EAD, government, Green Card, HTML, HTTP, IE, IM, Immigration, India, IP, ISP, it, leave, live, media, NY Times, Om, One, opinion, permanent resident, PHP, pr, Technorati, United States