Protect Your Online Revenue – Best Practices For Ensuring Your Web Applications Perform Across Browsers

NB: I do work for Gomez.

The methodology of the Search Performance Index is straightforward: A number of key search providers are selected and HTTP GET requests are sent that directly pull the results page of a query page that is searching for ‘New York’.

This is a simple process and one that is a close approximation of the actual search process that hundreds of millions of people perform every day.

alias                       Response Time  Success  Attempts  Success Rate
--------------------------  -------------  -------  --------  ------------
SEARCH - Google Blogsearch      0.2822545    27242     27244       99.9927
SEARCH - Google                 0.3151932    27228     27247       99.9303
SEARCH - Live (Microsoft)       0.3840097    27245     27246       99.9963
SEARCH - Indeed                 0.4112960    27240     27241       99.9963
SEARCH - Yahoo                  0.4574381    24175     24175      100.0000
SEARCH - Altavista              0.4592764    23922     23922      100.0000
SEARCH - Cuil                   0.6757475    23963     23967       99.9833
SEARCH - AOL                    0.7822945    23913     23913      100.0000
SEARCH - Ask                    0.9025220    24157     24163       99.9752
SEARCH - Technorati             0.9053472    27219     27234       99.9449
SEARCH - Amazon                 1.3251402    27245     27251       99.9780
SEARCH - Baidu                  1.7409345    23777     23799       99.9076
SEARCH - Blogdigger             1.8960633    25106     26354       95.2645
SEARCH - BlogLines              2.0238809    27233     27248       99.9450
SEARCH - IceRocket              2.1233684    24147     24177       99.8759
SEARCH - Blogpulse              2.4144131    27019     27247       99.1632

As can be seen in the data, there is a substantial degree of difference in the response time results, but the big three search providers (Google, Yahoo, and Microsoft Live) were in the top five. As well, all but three of the providers (Blogdigger, IceRocket and Blogpulse) had success rates (availability) of 99.9% or higher.

I don’t expect to see much in the way of change in these results, but I will post them for comparison each week.

This is not a bad or evil thing, considering that for at least 18 months, the articles that were hosted at those sites were duplicated here in a more manageable format.

For those who have come looking for the content from those sites, it is here. The search box in the right column can help you locate it.

But for those who would like a refresher, here is a list of the most popular articles on this blog, as selected by Web performance traffic.

Web Performance Concepts Series

• Web Performance, Part I: Fundamentals
• Web Performance, Part II: What are you calling ‘average’?
• Web Performance, Part III: Moving Beyond Average
• Web Performance, Part IV: Finding The Frequency
• Web Performance, Part V: Baseline Your Data
• Web Performance, Part VI: Benchmarking Your Site
• Web Performance, Part VII: Reliability and Consistency
• Web Performance, Part VIII: How do you define fast?
• Web Performance, Part IX: Curse of the Single Metric

Why Web Measurement Series

• Why Web Measurement, Part I: Customer Generation
• Why Web Measurement, Part II: Customer Retention
• Why Web Measurement, Part III: Business Operations
• Why Web Measurement, Part IV: Technical Operations

Web Performance Tuning

• The Dichotomy of the Web: Andy King’s Website Optimization
• Performance Improvement From Compression
• Baseline Testing With cURL
• Compressing Web Output Using mod_deflate and Apache 2.0.x
• Compressing PHP Output
• Using Client-Side Cache Solutions And Server-Side Caching Configurations To Improve Internet Performance
• Performance Improvement From Caching and Compression
• Compressing Web Output Using mod_gzip for Apache 1.3.x and 2.0.x
• mod_gzip Compile Instructions
• Hacking mod_deflate for Apache 2.0.44 and lower

IP Tracking

• Geographic IP database using PERL, PHP and MySQL — UPDATE: September 16 2008
• Hit Tracking with PHP and MySQL

The dynamic page was starting to push 20-25 seconds just for the Top 20 List. When I switched to the static list, times dropped to less than 1 second.

It’s always bad when a Web performance measurement site has poor performance.

There is a methodology statement explaining more about the index below the data table.

Please add any comments or questions to this post.

The switch to InnoDB was done because of the locking issues that were occurring during long queries, especially when doing ad-hoc analysis. The row-level (versus table-level) locking of InnoDB has removed most of these issues.

I have been seeing some strange behavior with the new engine. As a result of this, I will be re-starting the database engine twice a day. There should be no degradation, as this is simply a daemon re-start, not a machine re-start.

This downturn is a different beast. This means that you will need more than basic technical smarts to get through. To survive in the world of e-commerce for the next 18 months, here are some strategies you should take to heart.

• Clean up the cruft. Development and infrastructure changes are going to slow during the downturn – accept this. So do a code audit. Make sure that your applications aren’t calling files that no longer exist. Get rid of (backup and archive or Sarbanes-Oxley will bite your butt) old directories or application code that is no longer supported or maintained. In other words, do your Spring cleaning.

• Take care of the annoyances. In the rush to get code out or fix issues, little things that could help your site are often overlooked. Take the time now to optimize your images, minify your JS files, ensure your CSS files only contain styles that you use. Tune your SQL queries. Optimize your app code. All those things that got lost in the rush to get stuff out the door. As Microsoft and Apple have realized, people want performance, not more features.

• Make your site Browser Neutral. I work for a firm that allows me to view various sites with different OS/browser combinations easily and recommend to clients that they do the same. I hang my head when I look at my own blog and realize that it is not Browser Neutral. Take some time to investigate to how to eliminate this frustrating annoyance that makes people cranky and doubt your technical savvy.

• Get to know your visitors/customers. Unlike a ‘real’ store, you can’t step out onto the floor and talk to customers. But you can do this far more easily in today’s world than ever before. In the past, visitor analytics would have been the extent to which a company would have gone to determine information about their visitors/customers. Today, Twitter, Facebook, and all other manner of social sites make listening and talking to customers much easier. Just remember to be real.

• Get to know the people you work with. Just like getting to know your visitors/customers, you need to spend some time getting to know people in your organization. I know this sounds like manager-speak, but if you have contacts in Finance, Product Management, Operations, Admin, you are more likely to be able to more effectively due your job. If you understand the ebb and flow and stresses that are going on outside your little enclosed silo, you can place things in a larger perspective. While social-networking may work here, be careful to back it up with face-to-face contact.

• Get to know people! I made a number of contacts in the last downturn by taking my key interests (Web compression, Web caching) and turning them into sites, published articles, and one failed conference appearance (contact me if you want to hear more). Over the years these contacts have had me comment on their posts, edit their books, and keep them up-to-date on where I think the latest trends are headed. Going into this downturn, I have a whole group of new contacts that I am building on to do the same things. People aren’t bad, they’re just misunderstood.

• Read the fine business plan. I know these things are evil. They are in horrible management-speak. So don’t read it that way: Active read it. Make notes in the columns. Turn it into a MindMap (that’s what I did this year). Extract the meaning of what the goals of the business are and how the Web site fits into it. If you understand what the rest of the business is doing and what their challenges are, your problems will have context!

• Know what the next big thing is. While the objective of the organization in a downturn is to try and be more efficient with fewer resources, remember that you have to take time (whatever you have left) to know what’s coming next that could affect your Web site. Read the tech news. Understand the fundamentals that are driving the latest ideas. Keep on top of security. Sometimes a side-project that uses a technology that is new can become a bigger part of your site when resources free up.

I know these are general, but when this downturn started, I looked back on what worked for me during the last downturn. These are a few of the things that got me through, kept me busy, and helped me make an impact on the company and industry I am in.

These things also don’t hurt your reputation, as you become known as someone who understands the industry, the business, and the customers.

These mark my attempt to move from a purely technical analysis of providing a solution to the problem at hand. These posts move my approach to one that examines the issues from a higher perspective, one where I can place the singular issues that face people in the areas of Technical Operations and Business Operations into a cross-industry, global Web performance context.

These posts highlight some of the core concepts which need to be re-examined, especially within the reality that faced by all online businesses right now: tightening budgets, static or contracting staffing, and the need to not lose the performance edge.

I hope that these nine posts help you reconsider how you examine Web performance data when you are trying to maintain and grow your business in these troubled times.

1. Web Performance, Part I: Fundamentals
2. Web Performance, Part II: What are you calling ‘average’?
3. Web Performance, Part III: Moving Beyond Average
4. Web Performance, Part IV: Finding The Frequency
5. Web Performance, Part V: Baseline Your Data
6. Web Performance, Part VI: Benchmarking Your Site
7. Web Performance, Part VII: Reliability and Consistency
8. Web Performance, Part VIII: How do you define fast?
9. Web Performance, Part IX: Curse of the Single Metric

The issue that I see when I work with online firms is that the first item to be on the chopping block is the performance budget. Performance appears to be equated with expense.

What name I have in the arena of Web performance that I do have I made during the last economic slowdown in 2000-2002 by writing and evangelizing some very simple facts:

1. HTTP compression is easy to set up and can reduce the bandwidth that your text files require to download. This is even more important as more and more companies build complete applications in Javascript.
2. Allow others to cache what is cacheable. Offload your site to edge and corporate servers, and let them do the heavy lifting.
3. Enable HTTP persistence. Fewer TCP connection requests make downloads run more effectively and take advantage of the benefits of the efficiencies of built into TCP.

Steve Souders’ book reminds people that designers can help this process by designing a proper page. Andy King’s book (my review) reminds people that Business and Technical Operations need to work together to reach the same goal.

So, rather than seeing this time of contraction as a challenge, take it as a chance to solidify your performance position. Retrench, re-examine, and reveal your performance issues. Integrate Web performance into the day-to-day operations of the entire business, and work within your current configuration.

The companies that learn to work more effectively and efficiently with a reduced budget and fewer people will come out on the other side as those poised to take advantage of the opportunities that are still out there.

Three of the most contentious words, and most contentious acronym, in the technology sector. Arguments are had, suits are filed, and relationships broken and strained as a result of this single concept.

How can something seemingly simple as setting an agreed upon level of service delivery be so problematic and misunderstood?

The word agreement is the key to the problem. SLAs assume that all parties understand and agree of the level of service. And how that information is to be reported. And who is responsible for reporting the data. And how long you have to file grievances. And who handles problems. And…well, lawyers are involved.

As Guy Kawasaki states regarding the lies of venture capitalists: there is no such thing as a vanilla term sheet.

There is also no such thing as a vanilla SLA. A company that tries to present you with a standardized SLA is trying to pull something over on you.

Some rules about SLAs.

1. The vendor does not define the SLA. If the vendor selling the product tells you, the customer, what your expected level of service is, then they don’t care about you. Find another vendor.
2. The customer does not define the SLA. If the customer tells you that they cannot sign an SLA unless you, the vendor, agree to their conditions, walk away from the deal.
3. An SLA is not an SLO. Service Level Objectives are the targets of success defined by both parties within the SLA. These numbers, however, are not the alpha and the omega of an SLA
4. A customer-initiated penalty condition is always in the vendors favor. If the vendor states that the client must initiate the SLA grievance conversation when SLOs are violated, then the vendor is assuming that you are not looking at the data.
5. SLOs should never be based on single, aggregated metrics from the data. If some bozo tries to say that they provide 99% availability and 3 second average performance, walk away. That is not an SLO.
6. SLAs are not set in stone. If something is not working, or if targets change, or anything changes, then the parties have to be willing to sit down on a schedule (defined in the SLA) and renegotiate their SLA.
7. The vendor and the customer have transparent access to the data used for the SLO. If the ccustomer cannot see the data that the vendor is using in the SLO anytime it wants, there will always be a level of mistrust. If you like having all your customers mistrust you, this is a great strategy.
8. The Problem and Issue Management processes are clearly defined. When something bad happens, or a change needs to be made, the customer and the vendor have to have very clearly defined roles in the process. Responsibility and trust. Do you have that in your current SLA?
9. The customer and the vendor decide when a problem or issue is resolved. It is not up to one side in an SLA to decide when an issue or problem is resolved. As there are likely penalties involved the longer the abnormal state exists, the customer has a vested interest in quick resolution. As there is likely lost revenue on the table, the customer has the same interest. But the customer also has the seemingly unreasonable idea that this will never happen again, it will be clearly documented, and that getting the right solution is better than getting a solution.
10. Communication is the key to a good SLA. In the 9 previous points, the emphasis is on communication, the sharing of information. Current SLAs seem to be designed to hide information from each side, and only release it under the most dire situation. People talk. The information will get out. You want your well-crafted brand to implode because you have a reputation as sneaky and untrustworthy?

I’ve likely missed many of the key points, but these are the ones that I see, from both sides of the field, on a pretty regular basis.

In the end, an SLA is not simple. It is not standardized. It is not defined by one side or the other. It is a negotiated treaty of behavior that, in the end, defines the daily operational relationship between two organizations. If you enter an SLA process with both sides trying to find the best way to work together in the long term, there is a good chance that the SLA will be easier than if you go in as stone-cold adversaries.