This week, I have been discussing the different approaches to blog analytics that can be used to determine what posts from a blog’s archive are most popular, and whether a blog is front-loaded or long-tailed. The thesis is that it’s not always what the words in the blog are that are important.

In a guest post this morning at ProBlogger, Skellie discusses how the value of social media visitors is different and inherently more complex than the value of visitors generated from traditional methods, such as search and feedreaders. Her eight points further support my ideas that the old advertising models are not the best suited for the new blogging world.

Stepping away from the existing advertising models that have been used since blogging popularity exploded in 2005 and 2006, it is clear that the new, interactive social web model requires an advertising approach that centers on community and conversation, rather than the older idea of context and aggregated readership.

The Current Model

Current blog advertising falls into two categories:

1. Contextual Ads. This is the Google model, and is based on the ad network auctioning off keywords and phrases to advertisers for the privilege of seeing their ad links or images appear on pages that contain those words or phrases.
2. Sponsored Ads. Once a blog is popular enough and can prove a well-developed audience, the blogger can offer to sell space on his blog to advertisers who wish to have their products, offerings or companies presented to the target audience.

In my opinion, these two approaches fail blog owners.

Contextual ads understand the content of the page, but do not understand the popularity of the page, or its relationship to the popularity of other pages in the archive.Contextual ads lack a sense of community, a sense of conversation. While the model has proven successful, it does not maximize the reach that a blog has with its own audience.

Sponsored ads understand the audience that the blog reaches, but do not account for posts that draw the readers’ attention for the longest time, both in terms of time spent reading and thinking about the post as well as over time in an historical sense. The sponsored ad model assumes that all posts get equal attention, or drive community and conversation to the same degree.

The New Model

In the new model, more effective use of visitor analytics is vital to shaping the type and value of the ads sold. Studying the visitor statistics of a blog will allow the owners to see whether the blog is, in general, front-loaded or long-tailed.

If the blog has a front-loaded audience, the most recent posts are of higher value and could be auctioned of at higher prices. In order for this to work, both the ad-hoster and the advertiser would have to agree to the value of the most recent posts using a proven and open statistical analysis methodology. In the case of front-loaded blogs, this analysis methodology would have to demonstrate that there is a higher traffic volume for posts that are between 0-3 days old (setting a hypothetical boundary on front-loading).

For blogs that are long-tailed, those posts that continue to draw consistent traffic would be valued far more highly than those that fall out into the general ebb and flow of a bloggers traffic. These posts have proven historically that they appear highly in search results and are visited often.

In addition to the posts themselves, the comment stream has to be considered. Posts that generate an active conversation are farmore valuable those that don’t. Again, showing the value of the conversation is reliant of the ability to track the numbers of people in the conversation (through Disqus or some other commenting system).

This model can be further augmented by using a tool like Lookery that helps to clearly establish the demographics of the blog audience. Being able to pinpoint not only where on a blog to advertise but also who the visitors are who view those page, provides a further selling point for this new model and helps build faith in the virtues of a blog that sells space using this new, more effectively targeted advertising pricing structure.

Now, I separate the front-loaded and long-tailed blogs as if they are distinct. Obviously these categories apply to nearly every blog as there are new posts that suddenly capture the imagination of an audience, and there are older posts that continue to provide specific information that draws a steady stream of traffic to them.

Summary

This is a very early stage idea, one that has no code or methodology to support it. However, I believe that the current contextual advertising model, one based solely on the content of the post, is not allowing the content creators and blog entities to take advantage of their most valuable resource – their own posts and the conversations that they create.

I also believe that blog owners are not taking advantage of their own best resource, Web analytics, to help determine the price for advertising of their site. Not all blog posts are created or read equally. Being able to very clearly show what drives the most eyeballs to your site is a selling point that can be used in a variable-price advertising model.

By providing tools to blog owners that intimately link the analytics they already gather and the advertising space they have to sell, a new advertising model can arise, one that is uniquely suited to the new Web. This advertising model will be founded in the concepts of conversation and community, providing more discretely targeted eyeballs to advertisers, and higher ad revenues to blog owners and content creators.

UPDATES

Appears that BuzzLogic has already started down this path. VentureBeat has commentary here.

Since I started self-hosting this blog again on August 6 2008, I have been trying to find more ways to pull traffic toward the content that I put up. Like all bloggers, I feel that I have important things to say (at least in the area of Web performance), and ideas that should be read by as many people as possible.

As well, I have realized that if I invest some time and effort into this blog, it can be a small revenue source that could get me that much closer to my dream of a MacBook Pro.

The Analysis

In a post yesterday morning, Darren Rowse had some advice on when the best time to release new post is. Using his ideas as the framework, I pulled the data out of my own tracking database and came up with the chart below. This shows the page view data between September 1 2007 and September 15 2008 based on the day of the week vistors came to the site.

Using this data and the general framework that Darren subscribes to, I should be releasing my best and newest thoughts in a week on Monday and Tuesday (GMT).

After Wednesday, I should release only less in-depth articles, with a focus on commentary on news and events. And I must learn to breathe, as I suffer from an ailment all to common in bipolars: a lack of patience.

A new post doesn’t immediately find its target audience unless you have hundreds or thousands (Tens? Ones?) of readers who are influential. If you are luckyin this regard, then these folks will leave useful comments, and through their own attention, help gently show people that a new post is something they should devote their valuable attention towards.

It takes a while for any post to percolate through the intertubes. So patience you must have.

Front-loaded v Long-tailed

Unless, of course, your traffic model is completely different than a popular blogger.

The one issue that I had with Darren’s guidance is that it applies only to blogs that are front-loaded. A front-loaded blog is one that is incredibly popular, or has a devoted, active audience who help push page views toward the most recent 3-5 posts. Once the wave has crested, or the blogger has posted something new, the volume of traffic to older posts falls off exponentially, except in the few cases of profound or controversial topics.

When I analyzed my own traffic, I found that the most of my traffic volume was aimed toward posts from 2005 and 2006. In fact, more recent posts are nowhere near as popular as these older posts. In contrast to the front-loaded blog, mine is long-tailed.

There are a number of influential items in my blog which have proven staying power, which draw people from around the world. They have had deep penetration into search engines, and are relvant to some aspect of peoples’ lives that keeps pulling them back.

Summary

I would highly recommend analyzing your traffic to see it is front-loaded or long-tailed. I know that I wish that this blog  was more front-loaded, with an active community of readers and commentators. However, I am also happy to see that I have created a few sparks of content that keep people returning again and again. If your blog is  long-tailed, then when you post becomes far less relevant than ensuring the freshness and validity of those few popular posts. Ensure that these are maintained and current so that they remain relevant to as many people as possible.

Recently there was an outage at a hit-tracking vendor I was using to track the hits on my externally hosted blog, leaving me with a gap in my visitor data several hours long. While this was an inconvenience for me, I realized that this could be mission critical failure to an online business reliant on this data.

To resolve this, I used the PHP HTTP environment variables and the built-in function for converting IP addresses to IP numbers to create my own hit-tracker. It is a rudimentary tracking tool, but it provides me with the basic information I need to track visitors.

To begin, I wrote a simple PHP script to insert tracking data into a MySQL database. How do you do that? You use the gd features in PHP to draw an image, and insert the data into the database.

header ("Content-type: image/png");

include("dbconnect_logger.php");
$logtime = date("YmdHis");
$ipquery = sprintf("%u",ip2long($_SERVER['REMOTE_ADDR']));

        $query2 = "INSERT into logger.blog_log values \
               ($logtime,$ipquery,'$HTTP_USER_AGENT','$HTTP_REFERER')";
        mysql_query($query2) or die("Log Insert Failed");

mysql_close($link);

$im = @ImageCreate (1, 1)
or die ("Cannot Initialize new GD image stream");
$background_color = ImageColorAllocate ($im, 224, 234, 234);
$text_color = ImageColorAllocate ($im, 233, 14, 91);

// imageline ($im,$x1,$y1,$x2,$y2,$text_color);
imageline ($im,0,0,1,2,$text_color);
imageline ($im,1,0,0,2,$text_color);

ImagePng ($im);
?>

DROP TABLE IF EXISTS `blog_log`;
CREATE TABLE `blog_log` (
  `date` timestamp NOT NULL default '0000-00-00 00:00:00',
  `ip_num` double NOT NULL default '0',
  `uagent` varchar(200) default NULL,
  `visited_page` varchar(200) NOT NULL default '',
  UNIQUE KEY `date` (`date`,`ip_num`,`visited_page`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Data should begin flowing to your database immediately. This sample snippet of code will allow you to pull data for a selected day and list each individual hit.

$query1 = "SELECT
                bl.ip_num,
                DATE_FORMAT(bl.date,'%d/%b/%Y %H:%i:%s') AS NEW_DATE,
                bl.uagent,
                bl.visited_page
        FROM blog_log bl
        WHERE
                DATE_FORMAT(bl.date,'%Y%m%d') ='$YMD'
		and uagent not REGEXP '(.*bot.*|.*crawl.*|.*spider.*|^-$|.*slurp.*|.*walker.*|.*lwp.*|.*teoma.*|.*aggregator.*|.*reader.*|.*libwww.*)'
        ORDER BY bl.date ASC";

print "<table border=\"1\">\n";
print "<tr><td>IP</td><td>DATE</td><td>USER-AGENT</td><td>PAGE VIEWED</td></tr>";
while ($row = mysql_fetch_array($result1)) {
        $visitor = long2ip($row[ip_num]);
        print "<tr><td>$visitor</td><td nowrap>$row[NEW_DATE]</td><td nowrap>$row[uagent]</td><td>";

	if ($row[visited_page] == ""){
    	    print " --- </td></tr>\n";
	} else {
    	    print "<a href=\"$row[visited_page]\" target=\_blank\">$row[visited_page]</a></td></tr>\n";
	}

}

mysql_close($link);

For my own purposes, this is an extension of the Geographic IP database I created a number of years ago. This application extracts IP address information from the five IP registrars, and inserts it into a database. Using the log data collected by the tracking bug above and the lookup capabilities of the Geographic IP database, I can quickly track which countries and ISP drive the most visitors to my site, and use this for general interest purposes, as well as the ability to isolate any malicious visitors to the site.

I have set up a test to check the performance of the CoralCDN network against that of the origin server. You can view the comparative results here.

The tests used the base HTML document of this blog as the target.

The results so far indicate that there is a slight performance penalty when using CoralCDN in an ad hoc manner. They do offer continuous CDN services, and these likely provide better overall service under normal conditions.

However, it is likely that in situations where server load or traffic volumes increase substantially, the distributed performance system, even in an ad hoc manner, would save your bacon.

I will watch these tests over the next few days to see if any unique performance patterns appear.

This paper is an extension of the work done for another article that highlighted the performance benefits of retrieving uncompressed and compressed objects directly from the origin server. I wanted to add a proxy server into the stream and determine if proxy servers helped improve the performance of object downloads, and by how much.

Using the same series of objects in the original compression article[1], the CURL tests were re-run 3 times:

1. Directly from the origin server
2. Through the proxy server, to load the files into cache
3. Through the proxy server, to avoid retrieving files from the origin.[2]

This series of three tests was repeated twice: once for the uncompressed files, and then for the compressed objects.[3]

As can be seen clearly in the plots below, compression caused web page download times to improve greatly, when the objects were retrieved from the source. However, the performance difference between compressed and uncompressed data all but disappears when retrieving objects from a proxy server on a corporate LAN.

Instead of the linear growth between object size and download time seen in both of the retrieval tests that used the origin server (Source and Proxy Load data), the Proxy Draw data clearly shows the benefits that accrue when a proxy server is added to a network to assist with serving HTTP traffic.

	MEAN DOWNLOAD TIME
Uncompressed Pages
Total Time Uncompressed — No Proxy	0.256
Total Time Uncompressed — Proxy Load	0.254
Total Time Uncompressed — Proxy Draw	0.110
Compressed Pages
Total Time Compressed — No Proxy	0.181
Total Time Compressed — Proxy Load	0.140
Total Time Compressed — Proxy Draw	0.104

The data above shows just how much of an improvement is gained by adding a local proxy server, explicit caching descriptions and compression can add to a Web site. For sites that do force a great of requests to be returned directly to the origin server, compression will be of great help in reducing bandwidth costs and improving performance. However, by allowing pages to be cached in local proxy servers, the difference between compressed and uncompressed pages vanishes.

Conclusion

Compression is a very good start when attempting to optimize performance. The addition of explicit caching messages in server responses which allow proxy servers to serve cached data to clients on remote local LANs can improve performance to even a greater extent than compression can. These two should be used together to improve the overall performance of Web sites.

[1]The test set was made up of the 1952 HTML files located in the top directory of the Linux Documentation Project HTML archive.

[2]All of the pages in these tests announced the following server response header indicating its cacheability:

Cache-Control: max-age=3600

[3]A note on the compressed files: all compression was performed dynamically by mod_gzip for Apache/1.3.27.

Apparent using HTTP compression alongside HTTP/1.1 will cause certain versions of MSIE 6.0 to implode. [here]

I personally think this was because the NSA power shortage was making it too hard for the spooks to snoop on compressed Web traffic. [here]

Via: Port80 Software

PS: No, I won’t turn off compression because Microsoft did something really stupid.

Technorati Tags: Internet+Explorer, Internet+Explorer+6.0, MSIE, Port80+Software, HTTP+Compression, Web+Performance, Microsoft, NSA

Yup, we made it to Victoria. It was a 23-hour marathon that involved:

1. A shuttle service that made a change at the last second and saw us almost miss our flight. Dragon dude will be dishing out some justice about that soon.
2. A wildcat strike by air traffic controllers in Chicago that the pilots blamed on “weather”.
3. A motion sick kid from 20 minutes before landing in Seattle until we got to Grandma’s house.
4. Getting in the slow lane at the Douglas Border Crossing.

And then today, I blew up my father-in-law’s router doing an upgrade. had to do a downgrade to get it to work, but i may be buying him a new one.

Tomorrow, I will be rested and want to do something besides sleep and try and keep in touch with the world.

Technorati Tags: Victoria, BC, Canada, Seattle

Ok, this is a weird set of data, gathered by our friends over at the Internet Traffic Report

Any of you black-shirted, BGP-lovin’, AS-hoarding packet geeks out there want to comment on what happened on or about Oct 27-29, 2005? I am sure the rest of us would LOVE to know.

Technorati: bgp, Internet, Routing, internet performance

IceRocket: bgp, Internet, Routing, internet performance

Jeremy Zawodny asked the community for stats on where their search engine referrals were coming from.[here]

CLICK IMAGE

Google is crushing all other engines. However, I have noticed that my traffic has dropped substantially since the new Page Rank indexing began this week. Wonder what’s up there…

For all you local vloggers, there is a vlogging mini-conference coming up in Worcester, MA.

When: Monday, October 3rd, 6 p.m.
Who: You and your interested friends
What: “Meet the Vloggers” Worcester
Why: To learn more about videoblogging and build community
Directions to WPI: http://www.wpi.edu/About/Visitors/directions.html
Campus map to find the Campus Center:

http://www.wpi.edu/About/Visitors/Images/walkingmap.pdf

(The Campus Center is #6, behind Alumni Gym (#3) and next to Olin Hall
(#23). The star is where they are building a new Admissions building and the
circle in the center of campus is a fountain. These are just mentioned as
landmarks.)

Jan McLaughlin, a videoblogger (vlogger) from New York, will be visiting and
we will be presenting about videoblogging. This will include everything from
what equipment to use to how to drive traffic to your site, how to build
community through videoblogging and how to subscribe to news feeds using RSS
so you can watch other peoples’ videos. This will be a fun, interactive
presentation. I hope you can make it.

Thanks to Carl Weaver for the info.