How often does a recommendation algorithm get it right?

With artificial intelligence, recommendation and reminder algorithms invading most web based applications and services, I often wonder how often they actually get it right? As you probably know I love technology and will always be trying out new things and find out how they can potentially impact our lives.

Algorithms that give us recommendations based on our previous searches or behaviour, have been programmed to track past behaviour. Typically it is based on a trackable action ranging from search to view to click to buy. In my working world research is often part of the game, so some of the recommendations I receive are very relevant to projects, but a million years away from my personal preferences. In the world of learning I have often had to deep dive into a topic and find out the basics in order to design something of value. So in my case, the search based algorithms typically are off the mark for me personally.

Then when it comes to actually buying recommendations, the best example in my world is Amazon. It does show me relevant examples, because typically I will only buy what I want and not necessarily client related materials. But I also often receive recommendations of items I have already bought, which I find mildly amusing. Surely the same algorithm should be able to trace my old buys?

It reminds me of my early adventures into the world of VB programming. One of the lecturers made the comment about GUI, garbage in garbage out and in my mind this applies to todays recommendation algorithms. If you are not sure what will give the best results for your customers, ask them what they value and what they would actually buy for themselves. In my own experience, my buying behaviour is much more a reflection of my true self than for example my search patterns, which can be vastly different. I personally wonder if the creators of these engines mostly started with the view of what they want to track, rather than what the client would want to receive.

When you then enter multiple people into the equation, as in the case of our Tivo our television show recommender, it becomes really funny. We actually watch for the red buttons to jump on, to find out what exactly he thinks we need to be watching today. Funny enough, he can surprise us with great choices. Most often we have to throw out the majority of his great finds. We have a few series we have set him up to record, but really the accuracy is very hit and miss. Then my partner would watch sports, I watch news, technology and also very girly programs and dance related movies. We both like movies but from quite different genres. So no matter what Tivo has a difficult task ahead.

In our gamification design work the requests for some of this technology is slowly but surely entering the market. Some engines are working on integrating it out of the box, others are taking a wait and see approach. In my view the same design principle applies, you have to find out what is important for your customer to get the algorithm right.


Leave a comment

Our Solutions