One hundred and seventy-two million, three hundred and thirty-eight thousand; this is the number of unique websites on the Internet, as estimated by Netcraft, a well-respected UK based company. This is a staggering number of ‘virtual destinations,’ (approximately 1 website for every 39 people in the world) and even when you consider that these sites are spread across various languages and functions, there are still millions of websites that are relevant to people across the globe.

The truth is that the Internet is so vast that no living person could physically keep abreast of it all – if you tried to view a million websites for 1 minute each, it would take you over 3 years of waking life to see it all…and by then it would likely be outdated. The focus must then become, how does one find on the Internet, only those things that are relevant to one’s specific interests and needs, i.e. valuable content.

The traditional answer to this has been to search for it, after all, who knows your needs better than you?  The problem with this is two-fold; although Google, Yahoo, MSN and other search engines use sophisticated algorithms to find the ‘right’ content, good websites can still be lost in the obscurity of the internet if their owners don’t have the right publicity strategy or technical know-how to boost the website to its ‘fair market’ level of web traffic. This inefficiency may be further compounded by the end-user’s misuse of search strings, which results in less than optimal findings. Second, it can be inconvenient and time consuming to request information and then manually examine each link to validate it.  There has to be a better way to have the right information delivered to you, and my theory is that this will take shape in a new breed of ‘butler algorithms’ that deliver what you want, when you want it, and which will cut out the intermediary stage of having to look for it… or even ask for it.

These virtual butlers already exist in various forms, and what follows are a few rudimentary examples, but the true buttlers are yet to come.

Example #1: Amazon’s book recommendation system – “When in Rome…”

How does Amazon know what books to recommend to you, and how is it so accurate?  Amazon’s technical backend looks at the information you provide when you interact with the website (inputs, e.g. a search for ‘Wikinomics’), and then it compares these inputs with the purchase behaviour of other people (outputs/results) who bought the book you’ve expressed interest in, as well as other books. These ‘other books,’ across all Amazon users, are counted up and the top tallies are recommended to you. This system is helpful, but it doesn’t truly know you, it only knows how people like you behave and suggests that you might act in a similar fashion.

Example # 2: the recently released Digg recommendation system – “Knowing you, knowing your peers”

For those unfamiliar with Digg, users submit their favourite webpages on the Internet, which are then viewed, and voted up or down a ‘best of’ list by other members of the community. Similar to Amazon’s system, Digg implies that what others like, you will like, and normally the quality of content is quite good. The problem is that over 16,000 webpages are put up for vote every day and although this filtered down ‘best of’ the Internet is a farcry from the 172 million sites I mentioned earlier, it’s still too much information for people to sift through every day; the result is a net inefficiency in voting because people vote for the best of what they see (in a limited timeframe), rather than the best of what is actually there. In order to manage this high volume of information, the Digg team has developed a recommendation system that knows you; it knows what webpages you rate highly and what you rate poorly, and from these inputs it can start to construct your virtual personality. The second step is to compare your virtual personality to other people with similar tastes, and at that point the system suggests webpages that they have found interesting, suspecting that you might as well.

It’s worth noting that StumbleUpon also learns about your personal interests, and when it delivers ‘stumbled’ information to you, it notes how you react to the content, and learns from this reaction for future reference.  Thfilter.com is also an upcoming media delivery system that aims to do something similar and ‘filter out’ media that it knows you won’t be interested in.  These are good starts, but what I’m proposing is a little bit more complicated – imagine coming online to your virtual buttler, who knows that over the last 50,000 websites browsed (and counting), X% were news sites, Y% were blog sites, Z% were video or picture oriented, and so on.  Then imagine that this butler also knows when in the day you normally access those websites, how long you spend on each site, and how you normally branch out from those sites to others.  Although it may still be a ways off, I think that predictive formulas will one day be able to match relevant information to a potential suitor, streamlining Internet user’s experiences to create and enrich the virtual experience beyond its current form.

My question for you then is: What would your ideal virtual butler do?