A hallway conversation this morning brought up a very interesting point about the relationship between Web performance measurements and Content Delivery Networks (CDNs). When choosing between a Web performance measurement solution and a CDN, which service should come first?

Companies facing dire and obvious Web performance issues will want immediate results, leading them to fall into the CDN-First camp. Deploying a CDN will have a positive effect on response times, increase user satisfaction, and may even increase customer conversions, in the short term.

In six months, deeper questions may start to be asked. A core question that will need to be answered by CDN-First organizations will be “Are we using the CDN effectively and efficiently?“.

A company that makes the leap to CDN deployment without assessing the overall performance environment of their Web site may be faced with a situation where they can’t tell if they need more, less, or different CDN strategies in order to continue to succeed.

As a result of the buyers remorse that can result from the leap directly to a CDN, I highly recommend the Measurement-First approach when selecting a CDN.

To help you become an advocate for the Measurement-First approach, come to the table during the CDN discussions and ask three questions. The answers will allow your organization to make the best and most appropriate CDN decision.

1. Is the CDN necessary?

In most cases, the answer to this is a resounding yes. But what can happen with a sudden shift to the CDN is that a organization overlooks those things that they can do themselves to gain some initial performance improvements.

Baselining the existing site before deploying a CDN will allow items and elements that need to be improved to be clearly identified. In some cases, an organization can fix some of these on their own to improve performance before investing in a CDN. In other cases, measuring the performance of a site may clearly indicate that third-party content is responsible for the performance issues, which would likely not be fixed by a CDN deployment.

A Measurement-First policy helps clearly identify the geographies that have the worst performance before deploying the CDN. If performance in the US is acceptable, while performance in Europe or Asia-Pacific is intolerable, then the CDN deployment may initially be targeted to respond to the greatest pain first.

Understanding the current performance of your existing site can reduce the cost of the initial deployment and maximize the the long term effectiveness of the deployment.

2. Which CDN is best for us?

For a complex modern Web site, content comes in many different shapes, sizes, and formats. The thing is, so do CDNs. As I’ve discussed before, understanding what the CDNs vying for your business do and do well is as critical as the process of vetting their effectiveness compared to delivering the site yourself. The performance boost given to you site by a CDN may vary by region, leading your team to select one CDN for Europe and another for the Asia-Pacific region.

CDN performance can also vary based on the content you are asking them to accelerate. One CDN may be good at streaming media, while another may be better at static content (JS, CSS, Images, etc.), while yet another is better at accelerating the delivery of dynamic content.

Choose your CDN(s) based on what you need them to deliver. In some cases, one size does not fit all.

3. Is the CDN delivering?

This may look like a question for after the purchase has been completed and the solution deployed, but you will never know if the solution is working effectively unless you have a baseline of your performance before the deployment, and from your origin servers after deployment.

Measuring the performance of the CDNs under all conditions and from all perspectives (Datacenter, Last Mile, and from within the Browser) doesn’t stop with the selection of a CDN(s). It becomes even more critical once the CDN solution(s) is rolled into production in order to ensure that the level of service that was promised during the sales cycle is delivered once you become a customer.

Constantly validate the performance of the CDN-accelerated site with the performance of the non-accelerated origin site. Have regular meetings with, and channels of communication into, your CDN(s) to discuss not only existing performance, but how changes you and/or the CDN provider are planning may affect performance in the future.

Takeaway

CDNs are a critical component for any Web business that wants to scale and deliver services to a national or global audience. But selecting a CDN should come after you have a very strong understanding of the current performance of your own Web site.

After you have measured and identified the items you can do to improve your own performance, your team will have greater insight into the areas of your site where the services of a CDN(s) can have the greatest impact.

The Measurement-First approach to selecting a CDN will ensure that you select a set of services that exactly meets the unique performance challenges of your site.

It’s a rare Web site these days that hosts all of its own content. From the smallest blog to the largest retailer, Web sites farm out their images, streams, and pages to CDNs, and absorb feeds, ads, and data streams from any number of outside providers.

Effective Web performance demands that a site take responsibility for the entire site, not just the parts under direct local management. Why? Because customers see a problem with your site, not with a provider.

How can the performance all of the third-party content on a site be managed? Using the exactly the same strategies already place to manage the performance of local content.

Measure from the outside-in

Customers come from the Internet. That measuring the performance of a site from the perspective of visitors is being mentioned here should not be a surprise. Critical to this part of managing third-parties is the ability to see into the page and determine if there are performance issues requesting and transmitting data from third-parties.

In the first article of this series, I detailed a number of approaches to actively gathering performance data. This method, whether from the datacenter or the last mile, will provide the early warning signs that there is an issue with a third-party, and feeding this data into the performance issue management plan.

Measure from inside the browser

The network and application performance of a third-party page component is just the start of the process, as this is what it takes to get the object to the browser. But what if this object then launches a number of actions, or starts to render on the screen. This may lead to a whole different range of issues that are a blind spot when analyzing Web performance.

Measuring the performance of discrete page elements from within the visitors browser will provide deeper insight into what effects the customer sees and which third-parties will need to be approached in order to improve the overall Web performance of the site.

Have clear and useful SLOs and SLAs

Service level objectives and service level agreements are often thrown about whenever there is the suspicion that there is a Web performance issue. Using these documents and frameworks as a club to beat up partners with is counter-productive.

SLOs and SLAs should clearly detail:

• the performance expectations of the Web site owner
• the performance and delivery capabilities of the third-party provider

Guess what? Arriving at this in a way that doesn’t lead to resentment and mistrust on both sides requires open and honest discussion.

Share data

If Web site owners and third-parties are going to work together to ensure the most effective Web performance strategy possible, then data must flow freely. Vendors will need access to the same data that Web site owners have (and vice versa) in order to ensure that if an issue is detected, everyone can examine all of the available data, and solve the problem quickly.

Communication

A recurring and critical theme when establishing a culture of effective Web performance is communication. When working with third-parties, this is even more critical, as the performance culture of one organization may be completely different from another. The Web site owner may have one site of criteria that determines a Web performance issue, while the vendor has another, and unless these are understood, problems will occur.

Clear communication paths must be baked into the SLA. Named contacts or contact paths will be there, as will expected response times for inbound requests, and escalation procedures.

When there is a performance issue, both sides will need to be very clear about how each other will respond.

Takeaways

Third-party content on Web sites is a fact. It shouldn’t be a headache. Effective Web performance measurement strategies, shared sources of Web performance data, and clearly understood paths and methods of communication will make using third-party content less stress-inducing to everyone.

Content Delivery Networks (CDNs) are a key component to any Web performance strategy. If you examine the content from any large online business or media provider, it won’t take long to find the objects that these organizations have entrusted to CDNs to ensure faster delivery and a better user experience.

When working with CDNs, it is critical to understand some terms or concepts that you will be presented with. Each CDN will present them in it’s own unique way and using its own unique terminology. Having an understanding of the underlying concepts, you will be able to have discussions with CDNs that are more meaningful, and targeted on your needs.

The Massively Distributed Model

CDNs fall into one of two categories, the first being the massively distributed model. CDNs that use this method will demonstrate how they have hardware and caching content servers in almost every city and town of any size in the world. As well, they have their systems located on every major consumer network in order to ensure that they are as close to the end-user as possible.

The CDN everywhere model, while far-reaching and seemingly extremely effective does have its disadvantages. First, the CDN infrastructure relies on having extremely accurate maps of the Internet in order to direct visitors to the most proximate CDN server location. However, these maps are only truly effective when visitors use DNS servers that are on the same network that they are. Services such as OpenDNS and DNS Advantage can seriously effect the proximity algorithms of the distributed CDN by removing the key piece of localization information that they need to ensure that the best cache location is selected.

Also, as with any proxy caching methodology, this model relies on use. More popular items stay in the cache longer, while less popular items may be pushed aside or stored further upstream at parent caches for retrieval, adding a few extra milliseconds for the initial request. Also, new content has to be pushed out to the edge, and may take a few hours to be completely propagated.

The Massively Concentrated Model

CDNs that use this model rely on a smaller number of locations than the massively distributed model. However, these locations tend to be massive and incredibly well connected, relying on the concept that even if they are a few more hops away, their content is always there and ready for requests.

These sites have massive amounts of storage and rely on private networks to ensure that new content is immediately pushed out to the super-nodes as soon as it is added. And while they may be those extra few hops away, the performance difference may not be enough for the average site visitor to notice.

The obvious disadvantage of the massively concentrated model is that it is great for serving those places where there is a lot of traffic. However, in regions with less traffic, or less developed infrastructures, the fewer boots on the ground may begin to have an effect on performance.

Other CDN Concepts

Application Proxy

CDNs offer many institutions the ability to use their network for all incoming requests, even if they are for dynamic content that will require processing in the client datacenter. In these instances, the CDN acts as an application proxy, using its superior knowledge of routing and traffic patterns to move requests from the edge of the Internet back to the datacenter more effectively.

Remember: Just because the CDN is providing fast routing and delivery to the visitor, your application is still the bottleneck. Poor app design or slow queries will affect the application in exactly the same way that it would if the call was coming straight to your datacenter.

Traffic Acceleration

In certain circumstances, security and regulatory concerns completely eliminate the ability of a business to use the standard CDN model. Banks, government agencies, and health-care providers cannot store data in an environment whose security they cannot vouch for, no matter how many safeguards are put in place.

These organizations still need to be able to deliver a good customer experience, so there has to be a way to help accelerate their content without taking control of it. Traffic acceleration serves this purpose by using proprietary network protocol adaptations that remove some of the overhead associated with standard network protocols.

Content is intercepted at the datacenter and routed across private networks using the streamlined network protocols to an network location that is as close to the visitor as possible. Once it has reached the appropriate location, it is converted back to standard TCP and passed to the visitor.

The method above describes how a standard Web request works, but this can also be extended to true point-to-point VPNs with endpoints separated by great network and/or physical distances.

Validating the Claims

Any component of choosing or using a CDN is quantifying the effectiveness of the solution. The standard for many years has been the bake-off method of comparison. The prospect’s origin site is measured against the same site delivered by one or more CDNs. The CDN vendor with the fastest performance and the best price usually wins.

Before walking into a bake-off, come prepared. Turn your CDN bake-off into an episode of Iron Chef. Come to the table with the ingredients, and make the CDNs prepare a solution that meets your needs.

Measure Transactions

The standard base measurement that CDNs will use in a bake-off is single object(s) or page measurement. Your visitors do not just visit a single page, so ensure that the CDN has an effective solution that produces noticeable performance improvements across all the key functions of your site, including the secure components of the site, where the money is made.

Measure from the Edge

Backbone measurements are great for baselining and detecting operational issues that require a consistent and stable dataset. Your customers, however, do not have direct connections to high-priced datacenters with fat pipes.

The two CDN models will react differently to under certain circumstances, and this will appear in edge measurements. Measuring on the ground, from the ISPs that your customers use, will give you a clear sense of how much improvement a CDN will provide when compared to the performance of your origin datacenter.

The edge is messy, chaotic, and what your customers deal with everyday.

Understand the SLAs/SLOs

CDNs will always provide either service level agreement (SLA) with service level objectives (SLOs) stated in it. This topic is at once recognizable and about as well understood as 11 Dimensional Theoretical Physics.

I have written briefly about SLAs and SLOs before [here and here]. Do your research before you wade into this polite version of white-collar trench warfare.

Make sure you understand what the goal of the SLA is. Make sure that the SLOs are clear, measurable, valid, and enforceable. Then ensure that the method used to measure the SLOs is one that your organization can understand and can accept as valid.

Finally, ensure that the SLOs are reviewed monthly.

Takeaways

Understanding the foundational technology that underlies the CDNs you use or are considering using will help you make better decisions.

David Cancel and I have had sort of a passing vague, same space and thought process, living in the same Metropolitan area kind of distant acquaintance for about the same year.

About 2-3 months ago, he wrote a pair of articles discussing the efforts he has undertaken in order to try and offload some of the traffic to the servers for his new company, Lookery. While they are not current, in the sense that time moves in one direction for most technical people, and is compressed into the events of the past eight hours and the next 30 minutes, these articles provide an insight that should not be missed.

These two articles show how easily a growing company that is trying to improve performance and customer experience can achieve measureable results on a budget that consists of can recycling money and green stamps.

Measuring your CDN

A service that relies on the request and downloading of a single file from a single location very quickly realizes the limitations that this model imposes as traffic begins to broaden and increase. Geographically diverse users begin to notice performance delays as they attempt to reach a single, geographically-specific server. And the hosting location, even one as large as Amazon S3, can begin to serve as the bottleneck to success.

David’s first article examines the solution path that Lookery chose, which was moving the tag, which drives the entire opportunity for success in their business model, onto a CDN. With a somewhat enigmatic title (Using Amazon S3 as a CDN?), he describes how the Lookery team measured the distributed performance of their JS tag using a free measurement service (not GrabPERF) and compared various CDNs against the origin configuration that is based on the Amazon S3 environment.

This deceptively simple test, which is perfect for the type of system that Lookery uses, provided that team with the data they needed to realize that they had made a good choice in choosing a CDN and that their chosen CDN was able to deliver improved response times when compared to their origin servers.

Check your Cacheability

Cacheability is a nasty word that my spell-checker hates. To define it simply, it refers to the ability of end-user browsers and network-level caching proxies to store and re-use downloaded content based on clear and explicit caching rules delivered in the server response header.

The second Article in David’s series describes how, using Mark Nottingham’s Cacheability Engine, the Lookery team was able to examine the way that the CDNs and the Origin site informed the visitor browser of the cacheability of the JS file that they were downloading.

Cacheability doesn’t seem that important until you remember that most small firms are very conscious of the Bandwidth outlay. These small startups arevery aware when their bandwidth usage reaches 250GB/month level (Lookery’s bandwidth usage at the time the posts were written). Any method that can improve end-user performance while stilll delivering the service they expect is a welcome addition, especially when it is low-cost to free.

In the post, David notes that there appears to be no way in their chosen CDN to modify the Cacheability settings, an issue which appears to have been remedied since the article went up [See current server response headers for the Lookery tag here].

Conclusion

Startups spend a lot of time imagining what success looks like. And when it comes, sometimes they aren’t ready for it, especially when it comes to the ability to handle increasing loads with their often centralized, single-location architectures.

David Cancel, in these two articles, shows how a little early planning, some clear goals, and targeted performance measurement can provide an organization with the information to get them through their initial growth spurt in style.