Posts Tagged ‘performance improvement’

Web Performance: Your Teenage Web site

September 10th, 2008 by smp | Comments | Filed in Commentary, The Web, Web Performance, WebPerformance.Org, Work

It’s critical to your business. It affects revenue. It’s how people who can’t come to you perceive you.

It’s your Web site.

Its complex. Abstract. Lots of conflicting ideas and forces are involved. Everyone says they now the best thing for it. Finger-pointing. Door slamming. Screaming.

Am I describing your Web site and the team that supports it? Or your teenager?

If you think of your Web site as a teenager, you begin to realize the problems that your facing. Like a teenager, it has grown physically and mentally, and, as a result, thinks its an experienced adult, ready to take on the world. However, let’s think of your site as a teenager, and think back to how we, as teenagers (yeah, I’m old), saw the world.

MOM! This doesn’t fit anymore!

Your Web site has grown as all of your marketing and customer service programs bear fruit. Traffic is increasing. Revenue is up. Everyone is smiling.

Then you wake up and realize that your Web site is too small for your business. This could mean that the infrastructure is overloaded, the network is tapped out, your connectivity is maxed, and your sysadmins, designers, and network teams are spending most of your day just firefighting.

Now, how can you grow a successful business, or be the hip kid in school, when your clothes don’t fit anymore?

But, you can’t buy an entire wardrobe every six months, so plan, consider your goals and destinations, and shop smart.

DAD! Everyone has one! I need to have one to be cool!

Shiny.

It’s a word that has been around for a long time, and was revived (with new meaning) by Firefly. It means reflective, bright, and new. It’s what attracts people to gold, mirrors, and highly polished vintage cars. In the context of Web sites, it’s the eye-candy that you encounter in your browsing, and go “Our site needs that”.

Now step back and ask yourself what purpose this new eye-candy will serve.

And this is where Web designers and marketing people laugh, because it’s all about being new and improved.

But can you be new and improved, when your site is old and broken?

Get your Web performance in order with what you, then add the stuff that makes your site pop.

But those aren’t the cool kids. I don’t hang with them.

Everyone is attracted to the gleam of the cool new Web sites out there that offer to do the same old thing as your site. The promise of new approaches to old problems, lower cost, and greater efficiencies in our daily lives are what prompt many of us to switch.

As a parent, we may scoff, realizing that maybe the cool kids never amounted to much outside of High School. But, sometimes you have to step back and wonder what makes a cool kid cool.

You have to step back and say, why are they attracting so much attention and we’re seen as the old-guard? What can we learn from the cool kids? Is your way the very best way? And says who?

And once you ask these questions, maybe you agree that some of what the cool kids do is, in fact, cool.

Can I borrow the car?

Trust is a powerful thing to someone, or to a group. Your instinctive response depends on who you are, and what your experiences with others have been like in the past.

Trust is something often found lacking when it comes to a Web site. Not between your organization and your customers, but between the various factions within your organization who are trying to interfere or create or revamp or manage the site.

Not everyone has the same goals. But sometimes asking a few questions of other people and listening to their reasons for doing something will lead to a discussion that will improve the Web site in a way that improves the business in the long run.

Sometimes asking why a teenager wants to borrow the car will help you see things from their perspective for a little while. You may not agree, but at least now it’s not a yes/no answer.

YOU: How was school today? - THEM: Ok.

Within growing organizations, open and clear communication tends to gradually shrivel and degenerate. Communications become more formal, with what is not said being as important as what is. Trying to find out what another department is doing becomes a lot like determining the state of the Soviet Union’s leadership based on who attends parades in Red Square.

Abstract communication is one of the things that separates humans from a large portion of the rest of the animal kingdom. There is nothing more abstract than a Web site, where physical devices and programming code produce an output that can only be seen and heard.

The need for communication is critical in order to understand what is happening in another department. And sometimes that means pushing harder, making the other person or team answer hard questions that they think you’re not interested in, or that you is non of your business.

If you are in the same company, it’s everyone’s business. So push for an answer, because working to create an abstract deliverable that determines the success or failure of the entire firm can’t be based on a grunt and a nod.

Summary

There are no easy answers to Web performance. But if you consider your Web site and your teams as a teenager, you will be able to see that the problems that we all deal with in our daily interactions with teens crop up over an over when dealing with Web design, content, infrastructure, networks and performance.

Managing all the components of a Web site and getting best performance out of it often requires you to have the patience of Job. But it is also good to carry a small pinch of faith in these same teams, faith  that everyone, whether they say it or not, wants to have the best Web site possible.

Tags: , , , , , , , , , , , , , ,

Performance Improvement From Caching and Compression

October 3rd, 2006 by smp | Comments | Filed in Web Performance, WebPerformance.Org

This paper is an extension of the work done for another article that highlighted the performance benefits of retrieving uncompressed and compressed objects directly from the origin server. I wanted to add a proxy server into the stream and determine if proxy servers helped improve the performance of object downloads, and by how much.

Using the same series of objects in the original compression article[1], the CURL tests were re-run 3 times:

 

  1. Directly from the origin server
  2. Through the proxy server, to load the files into cache
  3. Through the proxy server, to avoid retrieving files from the origin.[2]

This series of three tests was repeated twice: once for the uncompressed files, and then for the compressed objects.[3]

As can be seen clearly in the plots below, compression caused web page download times to improve greatly, when the objects were retrieved from the source. However, the performance difference between compressed and uncompressed data all but disappears when retrieving objects from a proxy server on a corporate LAN.

uncompressed_pages

compressed_pages

Instead of the linear growth between object size and download time seen in both of the retrieval tests that used the origin server (Source and Proxy Load data), the Proxy Draw data clearly shows the benefits that accrue when a proxy server is added to a network to assist with serving HTTP traffic.

  MEAN DOWNLOAD TIME
Uncompressed Pages
Total Time Uncompressed — No Proxy 0.256
Total Time Uncompressed — Proxy Load 0.254
Total Time Uncompressed — Proxy Draw 0.110
Compressed Pages
Total Time Compressed — No Proxy 0.181
Total Time Compressed — Proxy Load 0.140
Total Time Compressed — Proxy Draw 0.104

The data above shows just how much of an improvement is gained by adding a local proxy server, explicit caching descriptions and compression can add to a Web site. For sites that do force a great of requests to be returned directly to the origin server, compression will be of great help in reducing bandwidth costs and improving performance. However, by allowing pages to be cached in local proxy servers, the difference between compressed and uncompressed pages vanishes.

Conclusion

Compression is a very good start when attempting to optimize performance. The addition of explicit caching messages in server responses which allow proxy servers to serve cached data to clients on remote local LANs can improve performance to even a greater extent than compression can. These two should be used together to improve the overall performance of Web sites.


[1]The test set was made up of the 1952 HTML files located in the top directory of the Linux Documentation Project HTML archive.

[2]All of the pages in these tests announced the following server response header indicating its cacheability:

Cache-Control: max-age=3600

[3]A note on the compressed files: all compression was performed dynamically by mod_gzip for Apache/1.3.27.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

GrabPERF: Main Page Performance Improvement

August 24th, 2006 by smp | Comments | Filed in GrabPERF, Linux: Server, Technology, Web Performance

One of the performance hits that the GrabPERF system has is the dynamic generation of the main page. The nature of the SQL calls and the underlying PHP makes it scale exponentially past a certain number of measurements.

Last night, Kevin Burton made a grand suggestion: generate a static page on a regular schedule.

Duh!

Today, I wrote the script that does this. The performance of the main page has adjusted accordingly.

GrabPERF Main Page Performance Improvement - Aug 24 2006

Yikes!

UPDATE: Ian Holsman reminded that if I use cURL, I can use the exiting PHP to build the pages without a PERL script.

I. AM. AN. IDIOT.

Now, bedtime.

Technorati Tags: , , , , , ,

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , ,

GrabPERF: State of the System, Feb 2006

February 19th, 2006 by smp | Comments | Filed in GrabPERF

If anyone wants to know why I am proud of GrabPERF, this graph should give you a clue.

GrabPERF Growth

Every hour, an aggregated value is produced for every test url. Counting up the Geometric Mean aggregations on a daily basis, the growth line is pretty amazing.

Doing a rough calculation, the system has grown from testing 40 urls to testing 104 urls.

But the true scope of this growth can only be seen by looking at the number of data insertions into the raw data table on a daily basis.

GrabPERF Measurements Per Day

Currently, with the four measurement agents (I turned down one of the Technorati agents today to relieve the strain on the database server), the system is handling nearly 300,000 data insertions a day. Not even in the same timezone with most of the major sites I measure, but when I think that this is a system that I designed, I am stunned. For an unfunded, not-for-profit, one-person effort, I am constant astounded by what this system can handle.

Other areas of note over the last year:

  • The Technorati-donated servers can now easily hold 60 days of detail data on a robust enterprise grade system, in a real datacenter
  • The basement datacenter is now closed
  • The new interface was created specifically to allow the system to grow and easily accommodate new features
  • People are now approaching me on a daily basis to have sites added, or to have data explained
  • GrabPERF went from one measurement agent to five agents in four locations, including a location in Europe (Portugal)
  • GrabPERF has been used in various places to serve as an indicator or motivator for performance improvement, including: Bloglines, Ping-O-Matic, Technorati, PubSub (1 and 2), Blogwise, and others.
  • A number of corporate and individual sponsors have stepped forward to support our efforts in many different ways: money, servers, measurement locations, commentary and critiques

I know that I have said it many times in person, or online, but thank you all. Those of you who use and support GrabPERF are the ones who continue to motivate me to make this system better on a daily basis.

Keep those cards and letter coming.

Technorati Tags: , , ,

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

BlogFlux Mapstats — Performance Improvement!

February 16th, 2006 by smp | Comments | Filed in GrabPERF

The team at BlogFlux laid on some major performance improvements to their Mapstats service last night.

BlogFlux Improvement -- Feb 16 2006

This is a fantastic improvement! I can’t wait to hear how they achieved it.

Technorati Tags: , , ,

Tags: , , , , , , , , , , , , , ,

GrabPERF: Search Index Weekly Results (Sep 12-18, 2005)

September 19th, 2005 by smp | Comments | Filed in smp

The weekly GrabPERF Search Index Results are in.

This Week’s Notes

  1. ERTW.com Agent location taken offline
  2. Addition of Google Blogsearch to the Index
  3. Numerous performance improvements to the GrabPERF interface

Week of September 12-18, 2005

TEST                           RESULT  SUCCESS  ATTEMPTS
--------------------------  ---------  -------  --------
PubSub - Search             0.2688096    99.82      6532
Google - Search             0.4013164    99.97      6532
Google Blogsearch - Search  0.5818507    98.60      4214
MSN - Search                0.6981630    99.83      6532
Yahoo - Search              0.7159974    99.95      6527
eBay - Search               0.8345692   100.00      6528
BlogLines - Search          1.0204595    99.95      6531
BestBuy.com - Search        1.1687228    99.97      6530
Feedster - Search           1.3112797    99.82      6531
Technorati - Search         1.3240335    99.95      6528
Amazon - Search             1.5195445    99.72      2481
Newsgator - Search          1.5823492    99.72      6529
Blogdigger - Search         1.7142475    99.97      6506
BENCHMARK RESULTS           2.0313721    99.50     76849
IceRocket - Search          4.2792600    98.79      6515
Blogpulse - Search          6.5226776    99.29      6522

These results are based on data gathered from two remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.

The measurements are for the base HTML document only. No images or referenced files are included.


Technorati: , , , , ,

IceRocket: , , , , ,

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , ,

Performance Improvements to b2evolution

September 16th, 2005 by smp | Comments | Filed in smp

I upgraded b2evolution yesterday to 0.9.1, which the developers claim had substantial performance improvements.

I believe them.

Kudos to the b2evolution team!

Tags: , , , , , , , ,

Ok, maybe the drama queen act doesn’t suit me…

July 28th, 2005 by smp | Comments | Filed in smp

You have to wonder about the resiliemcy of the human mind sometime; apparently as quickly as one dives into a deep funk, you get to bounce off the floor….

I’m Baaaaaaacccckkk!

If only to handle the flood from Dave Sifry’s post on Technorati’s performance improvement. [here]


Technorati: , , , , ,

IceRocket: , , , , ,

Tags: , , , , , , , , , , , , , , , , , , , , ,

Performance Improvement From Caching and Compression

April 30th, 2005 by smp | Comments | Filed in smp

This paper is an extension of the work done for another article that highlighted the performance benefits of retrieving uncompressed and compressed objects directly from the origin server. I wanted to add a proxy server into the stream and determine if proxy servers helped improve the performance of object downloads, and by how much.

Using the same series of objects in the original compression article[1], the CURL tests were re-run 3 times:

  1. Directly from the origin server
  2. Through the proxy server, to load the files into cache
  3. Through the proxy server, to avoid retrieving files from the origin.[2]

eries of three tests was repeated twice: once for the uncompressed files, and then for the compressed objects.[3]

As can be seen clearly in the plots below, compression caused web page download times to improve greatly, when the objects were retrieved from the source. However, the performance difference between compressed and uncompressed data all but disappears when retrieving objects from a proxy server on a corporate LAN.


Instead of the linear growth between object size and download time seen in both of the retrieval tests that used the origin server (Source and Proxy Load data), the Proxy Draw data clearly shows the benefits that accrue when a proxy server is added to a network to assist with serving HTTP traffic.

Uncompressed Pages

MEAN DOWNLOAD TIME
Total Time Uncompressed — No Proxy 0.256
Total Time Uncompressed — Proxy Load 0.254
Total Time Uncompressed — Proxy Draw 0.110

Compressed Pages

Total Time Compressed — No Proxy 0.181
Total Time Compressed — Proxy Load 0.140
Total Time Compressed — Proxy Draw 0.104

The data above shows just how much of an improvement is gained by adding a local proxy server, explicit caching descriptions and compression can add to a Web site. For sites that do force a great of requests to be returned directly to the origin server, compression will be of great help in reducing bandwidth costs and improving performance. However, by allowing pages to be cached in local proxy servers, the difference between compressed and uncompressed pages vanishes.

Conclusion

Compression is a very good start when attempting to optimize performance. The addition of explicit caching messages in server responses which allow proxy servers to serve cached data to clients on remote local LANs can improve performance to even a greater extent than compression can. These two should be used together to improve the overall performance of Web sites.


[1] The test set was made up of the 1952 HTML files located in the top directory of the Linux Documentation Project HTML archive.

[2] All of the pages in these tests announced the following server response header indicating its cacheability: Cache-Control: max-age=3600

[3] A note on the compressed files: all compression was performed dynamically by mod_gzip for Apache/1.3.27.

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Web Performance Evangelism Run Amok

December 7th, 2004 by smp | Comments | Filed in smp

I wanted to point you to an evangelist of the good kind that Scoble found — "Obi-Wan", the Prowler Knight. They come in all shapes and sizes.

One of the directors in our company keeps saying how impressed he was by a certain product evangelist he saw at a conference a few years ago. He sings high praises about my potential to do the same. I know I can — spent the last five years delving into the how-tos of Web performance, and have a bit of an opinionated streak to help me along.

Today, I am going to evangelize on Web performance.

The issue is that everyone has specific questions and nobody wants to think about the actual big picture. The biggest question an online has to ask is: "How do we make it fast, reliable, scalable, efficient and economic?"

Easy, right? Well, no actually. Big players in the online commerce world still have problems with this. Why? Why can’t they get it right?

Over the last few days, I have posted a couple of screenshots showing that Amazon, the online retailing poster child, has had 3 distinct and length outages. This is unheard of from them. However, they should be in seasonal lockdown at the moment. So I looked at some data I have access to last night. I know when the problem started, but don’t know the root cause. It is frightening that in the span of a single day, the internet leader is in the uncomfortable position of scrambling to decipher and resolve their problem during the busiest time of the year.

This doesn’t surprise me anymore. I just shrug my shoulders and say, loudly, for the umpteenth time that if someone had asked the right questions, followed the correct process, and accurately analyzed the data none of this would be happening.

I have said it before: I have tried. Look at a retailer like Amazon, and you must also look at Target — Target is completely wedded to the Amazon Infrastructure. Was Target part of the analysis of the data so that they could approve the system state freeze? The answer is likely no, and you know what? Target is likely going to collect a ton of paybacks from SLA infringements as a result of the Amazon outages.

At the beginning, I asked what does it take to achieve Web performance excellence. The answer is time and dedication. Online businesses have to either dedicate themselves to this, or sign on to partners who can.

Some big firms think that the traditional IT consulting firms can do it. What is their expertise in Web performance? How do they plan to validate and verify that the improvement plan they have outlined is actually meeting your business objectives? How will they help you manage your content, customer-tracking and ad providers?

Big IT consulting firms: Can you validate and verify that the performance improvements that you have implemented are economical? Are they efficiently resolving the issue? Who resolves problems?

How many consultant, engineers, developers and business managers does it take to fix a bad Web page?

Answer: I don’t know. Do you?

In the end, Web performance is no longer about response times and success rates. It is no longer about usability. It is no longer about hit tracking, processor utilization, SANs, and distributed content. We performance boils down to a single question:

"How do we make it fast, reliable, scalable, efficient and economic?"

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,