For FriendFeed has become my replacement for Google Reader, which I only visit occasionally now to see if there are blogs I need to add to my feed.

But, if you are going to replace a reader with FriendFeed, how do you manage the flow of content. While tools will likely improve over time, I have adopted a simple strategy.

1) Scan for items with obvious links

As I power through the front page of my feed, I look for items that are obviously links to longer articles. I can then decide if I want click through to that article. But rather than opening it in a new tab right in front of me, I use the wheel-click option in Firefox and open these articles in a background tab. This allows me to scan through the fees and read the articles when I want.

2) Read Twitter/indenti.ca/Jaiku/etc. last

Personal conversations come second for me. If there is a thread I am interested in, I will wheel click the Twitter page for the person and pick it up that way…or use Twitter Search. Being the kind of person who processes personal communications last makes this easier.

3) Use the FriendFeed interface as much as I can

If there was a way to open posts in a frame such as the way that video and images are embedded in FriendFeed, I would never go to anyone’s actual site. While that may be a feature of the future, the storage implementation for the FriendFeed team is potentially enormous - unless they choose to retrieve the content on the fly.

And finally…

4) Gripe about TinyURL, etc. links and how I don’t know where they lead

A great feature of the future for FriendFeed would be to translate obfuscated URLs to their base URLs in a rollover

And there you have it. Not the world’s most intense primer on using FriendFeed, but it works for me!

Tags: friendfeed, google reader, optimize, strategy, summize, tinyurl, twitter, twitter search

If you use GrabPERF on a regular basis, the somewhat flaky navigation method has become second nature to you. In fact, to circumvent some of the idiosyncrasies, you have probably bookmarked your favourite pages.

Yesterday, I broke your links.

When I redesigned GrabPERF in February 2006, I had just discover the require function in PHP, and decided to build the entire the structure using a single container page as the framework, and individual functions called using URL parameters.

As time went on, my own “brilliance” started to get in the way of maintaining and updating the code. It took me 10-15 minutes to figure out how I constructed pages, and then find the right code to fix or update.

Yesterday, I got completely fed up with this structure.

Now, all functions have their own unique pages, making maintenance a snap. And as an added benefit, I can now effectively track the usage of individual pages, so I know where to through development efforts.

Some of the changes.

http://grabperf.org/homepage.php?page=compare&test=2&tests%5B%5D=276&tests%5B%5D=277&tests%5B%5D=279&tests%5B%5D=280

becomes

http://grabperf.org/compare.php?test=2&tests%5B%5D=276&tests%5B%5D=277&tests%5B%5D=279&tests%5B%5D=280

http://grabperf.org/homepage.php?page=scatter&test=277&hours=2

becomes

http://grabperf.org/scatter.php?test=277&hours=2

I apologize for the confusion that this may cause, but in the long run, this will help me make the code better, and more robust.

Tags: book, content, design, GrabPERF, GrabPERF.org, HTTP, IE, IM, improvement, ISP, it, maintain, maintenance, Om, optimize, Pages, performance, PHP, pr, run, Technorati, update, web, Web Performance

This paper is an extension of the work done for another article that highlighted the performance benefits of retrieving uncompressed and compressed objects directly from the origin server. I wanted to add a proxy server into the stream and determine if proxy servers helped improve the performance of object downloads, and by how much.

Using the same series of objects in the original compression article[1], the CURL tests were re-run 3 times:

1. Directly from the origin server
2. Through the proxy server, to load the files into cache
3. Through the proxy server, to avoid retrieving files from the origin.[2]

This series of three tests was repeated twice: once for the uncompressed files, and then for the compressed objects.[3]

As can be seen clearly in the plots below, compression caused web page download times to improve greatly, when the objects were retrieved from the source. However, the performance difference between compressed and uncompressed data all but disappears when retrieving objects from a proxy server on a corporate LAN.

Instead of the linear growth between object size and download time seen in both of the retrieval tests that used the origin server (Source and Proxy Load data), the Proxy Draw data clearly shows the benefits that accrue when a proxy server is added to a network to assist with serving HTTP traffic.

The data above shows just how much of an improvement is gained by adding a local proxy server, explicit caching descriptions and compression can add to a Web site. For sites that do force a great of requests to be returned directly to the origin server, compression will be of great help in reducing bandwidth costs and improving performance. However, by allowing pages to be cached in local proxy servers, the difference between compressed and uncompressed pages vanishes.

Conclusion

Compression is a very good start when attempting to optimize performance. The addition of explicit caching messages in server responses which allow proxy servers to serve cached data to clients on remote local LANs can improve performance to even a greater extent than compression can. These two should be used together to improve the overall performance of Web sites.

[1]The test set was made up of the 1952 HTML files located in the top directory of the Linux Documentation Project HTML archive.

[2]All of the pages in these tests announced the following server response header indicating its cacheability:

Cache-Control: max-age=3600

[3]A note on the compressed files: all compression was performed dynamically by mod_gzip for Apache/1.3.27.

Tags: ads, apache, app, ARIN, bandwidth, cache-control, Cacheability, caching, client, compress, compression, control, corporate, data, EAD, extension, gzip, HTML, HTTP, IE, IM, improvement, IP, it, LAN, Linux, mod_gzip, Om, One, optimize, origin server, Pages, performance, performance improvement, pr, proxy, proxy server, run, server, the Origin, The origin server, traffic, web, wordpress

Normally when I discuss the performance of a page I am measuring using GrabPERF, it’s either good news (”you just got 5 times faster!”) or bad news (”your page hasn’t loaded in 6 months; you still there?”).

Today, something a little different: a question. What’s the question?

Why is the performance of a Technorati Blog (aka Traditional) Search so different from a Technorati Tag Search?

For those of you who have been around for a while, you know that Technorati allows you to search for results based on a Traditional search engine methodology, which is date-ranked, most recent first. It also provides a way to search through the user-defined tags that are appended to posts, or listed as category titles.

The issue that I have been seeing from my measurements is that Tag Searching is performance substantially worse than Traditional Search.

TRADITIONAL SEARCH

---

TAG SEARCH

What I need to understand from the Technorati team is the particular technical challenges that differentiate Traditional v. Tag Searching, because the difference in performance is astonishing.

And then there is the success rate of the Tag Search.

TECHNORATI TAG SEARCH SUCCESS RATE

When I examine the data, almost all of the errors on the Tag Search measurement are Operation Timeouts. I have set the GrabPERF Agent to time out when no response has come back for the server in 60 seconds. So, effectively 15% of the Tag Searches do not return data to the client in 60 seconds.

So, while the Traditional Search has been tuned and optimized, there appears to be much work left to make the Tag Search an effective and useful tool.

Technorati: technorati, search, tagging, seo

IceRocket: technorati, search, tagging, seo

Tags: app, ARIN, blog, Challenges, client, data, GrabPERF, GrabPERF.org, HTTP, IE, IM, it, measurement, media, normal, Om, optimize, performance, posts, pr, search, seo, server, Technorati

You know it’s going to be a good day when the DBA comes up to you and says:

“You know those really huge and stupid queries that we always yell at you about? Well, we have some database hardware in place and we thought of you as someone who could stress the system.”

Fast hardware. Fast Drives. And a query headed quickly towards the 2-3 million row count and 10 minutes of running time.

I love abusing hardware.

Tags: data, database, EAD, IE, IM, it, Om, One, optimize, run

Over the last two days I have been trying to optimize the performance of the Index Charts/Graphs. I think I have found the fix, and you should see results for these charts in the sub 1-second range.

To explain how this works, I have to open up the GrabPERF data model and let you folks peer under the hood.

The Daily Index graphs are created using two tables: hourly_site and data. hourly_site is a rollup produced asynchronously at the end of every hour that aggregates the Arithmetic and Geometric means, as well as a count of successful measurements. data contains over 30 days of raw measurement data which is purged at the end of each day.

All of the days in the daily graph, and all of the hours in the hourly graph are pulled from hourly_site. That is with the exception of the most recent day (daily) and the most recent hour (hourly). These are pulled dynamically from data to ensure that all current results appear on the right-hand side of the graph.

With this design, performance was awful. I have tweaked the code as much as I feel comfortable doing (I am an analyst, not a developer!), but it was slow. I increased the size of the query cache, but the boost only lasted for a while. Then I realized that the data I most cared about was being pushed out of the cache by all of the other cached data.

I have now set MySQL to cache queries only on demand. This reduces the load and means that only the results I care about, and which are accessed most frequently, are cached.

When I restart the database, the Daily Blog Search chart takes about 5.5 seconds to generate, and the hourly chart takes about 30 seconds. Yikes!

After this initial pain, both charts take less than one second to be generated.

I am still working on it, so let me know if you see any weirdness.

UPDATE: Seems that the first query of every hour pays the performance penalty, and all other queries will not.

Tags: app, blog, current, data, database, design, geometric mean, GrabPERF, IE, IM, it, measurement, mysql, Om, One, optimize, performance, pr, search, the other, update

Just noticed that Bob Wyman of PubSub dropped in to look at the GrabPERF data we are collecting. [here]

Although Bob’s pride is evident, I do caution that being the top of a performance index is simply one measure of success. There is the whole discussion of relevancy of results, but that is beyond the scope of a performance-related discussion, such as the one I am encouraging on this blog.

Some of the concepts that I have been collaborating on have just been released to the public by my employer. Extending the outdated concepts of comparing Web sites simply on speed and availability, we have created Consistency and Optimization indices. [here]

These destroy any hope that faster is always better. Having a consistent Web performance and highly optimized site design counts for more as I get older than raw speed does.

My analogy is simple, and comes from the late, great Hunter S Thompson:

In horse breeding, for instance, there is a definite risk in breeding two fast horses who are both a little crazy. The offspring will likely be very fast and also very crazy. So the trick in breeding thoroughbreds is to retain the good traits and filter out the bad.

The Kentucky Derby is Decadent and Depraved

What good is a fast Web site that is “very crazy” and highly inconsistent?

Congrats to Bob Wyman and the PubSub team. I hope that your success continues.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, PubSub

IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, PubSub

Tags: ARIN, benchmarking, blog, Consistency, data, design, GrabPERF, GrabPERF.org, HTML, HTTP, IE, IM, it, measurement, Om, One, optimize, performance, performance measurement, pr, Technorati, web, Web Performance, Web performance measurement, Web+performance

Ping-O-Matic. Weblogs.com.

Two of the sites that have seen surges in Web traffic recently.

My unsubstantiated theory is that as summer ends, Web traffic is seeing its usual surge, amplified by the new interest in blogs and Web 2.0 properties.

For the small companies who have effectively had the summer off, it’s time to re-adjust your capacity-planning estimates…upward.

Why? Rising oil prices will reduce travel. Disenchantment with movies will reduce going out. Gourmet take-home will reduce restaurant visitation. And increasing broadband access will increase demand for online services.

Be afraid…and get ready for more traffic. Lots more traffic.

Have you optimized your site for maximum performance yet?

Technorati: Web Performance, Capacity Planning, Web 2.0, SEO, traffic analysis

IceRocket: Web Performance, Capacity Planning, Web 2.0, SEO, traffic analysis

Tags: analysis, blog, broadband, EAD, HTTP, IE, IM, it, LAN, Om, online, optimize, performance, pr, seo, Technorati, traffic, web, web 2.0, Web Performance, Web+performance

Ok, I got through the neuro-chemical wall by doing some coding. It’s weird, but intense mental activity seems to give me enough of a boost for the meds to start working again.

I created two new graphs for GrabPERF. The performance of the Multi-Test Hourly chart is hideous, mainly because my 6 year-old writes better code than I do. What was the only solution I could come up with? A loop within a loop, with each inner loop running a SQL query that returns exactly 1 row of data, if there is data.

Ugh.

I got sucked into reading dooce for the first time. Got a question: does Heather every smile?

Back to your regularly scheduled Web performance rant later. Off to try and optimize some inner-looping madness.

Tags: data, EAD, GrabPERF, HTTP, IM, it, Om, optimize, performance, run, web, Web Performance

…and I have heard a rumour that David Allen is adding them to GTD, Merlin and Dave Winer are porting them to MacOSX, SOGrady is building them on Gentoo, and Scoble uses them on his tablet. Kathy Sierra is evangelizing how they make happier screwers (???) and Darren Rowse has pointed out how to optimize your layout to take advantage of Google ScrewSense.

Great. Thanks Seth. Now everyone will want Robertson Screws. One more thing Canadians will lose their exclusive superiority in. Losing hockey was bad enough. Now you want our screws.

Time to move back to Canada, where all cool things — including weather and excluding Nortel and Corel — start.

Seth Godin’s post

My Rant on the Cool Tools post

Technorati: tools, seth godin

Tags: app, blog, Canada, cool, Darren Rowse, Google, HTML, HTTP, IE, IM, it, Mac, Om, One, optimize, PHP, Technorati