From attention to personalization

This article was originally published in web 2.0 journal

Last week I wrote here about the need to work out the “architecture of attention.” This week I will focus on a specific application attention platform – personalization. My article last week generated really interesting and important discussion about the the value of attention capturing. We questioned the utility of attention and concluded that using attention to fuel personalization would be a great service that a lot of people can use.

This Friday, Michael Calore wrote a post in the Monkey Bites blog about new company called Idiomag. This company plans to launch the first personalized online magazine. How will it do that?

You can watch the screencast to get the details, but the basic idea is that their software turns attention and rating information into a personalized filter. This filter is then used to discriminate between interesting and not so interesting information.

I thought a lot about this problem for the last two years, particularly before I started adaptiveblue. So I’d like to share with you my thoughts on how personalization would actually work, and explain why the “attention information” is the necessary building block of the personalization.

We are running out of time

I am sure you feel this every day too – we have less and less time to get done more and more. The dilemma is how can we accomplish things and what we should be focusing on? Prioritizing becomes ever more important since we are constantly making tradeoffs between doing one thing and another.

Right now we do this manually. Everyday, for example, we sift through tens, if not hundreds, of news articles, trying to find the ones that matter to us. So far we have been fairly successful, but we are getting tired, because everyday we start from scratch and the amount of information keeps on increasing.

I have a simple metaphor for what we have become: we are information filters. Of course evolutionarily speaking we have always been that, but now the information filtering have moved from deep subconsciousness to the forefront of our mind. We think about information explicitly all the time. We are aware that we are information filters.

HAL, are you there?

Computers have been always bad at what we are good at and vice versa. One thing that computers are great at, is running the same algorithms over and over again. Computers are very good at data crunching, so whenever we encounter a problem of sifting through a lot of information, we generally tend to bring computers in. The question is then: why can’t we make computer choose what is important for us?

Well, to begin with, we can’t even trust our friends and family to do that, why would we trust a computer? This problem is tricky and sensitive, there is little tolerance for false positives here. But even though we are all very particular about bits that we choose to digest, we know deep-down that computers can help us be more effective, so then the question is: how?

Experience is the king

How do we ourselves choose the information? The magic in our heads does it based on what we have encountered in our past  –  10 years ago, a year ago, a week ago, yesterday and 10 minutes ago. The wonderful interplay of neurons creates a dynamic, evolving, intelligent filter that helps us decide what is important for us today. The key observation is that the filter evolves based on our experiences with the world.

So to succeed in building a computer program that can help us make decide what to pay attention to, we need to have this program experience, at least to the extend possible, what we are experiencing. It needs to know what we like and what we do not like, otherwise it just can’t do the job. The idea that we will spend time telling this program what we like is not going to work. We are way too busy to constantly update and correct the computer, we just do not have the time.

Instead, the ‘learning’ of our likes and dislikes needs to be integrated into our daily life. As we interact with the information, with every click we reveal our preferences. The successful personalization technology needs to understand that and seamlessly plugin into this process.  Instead of continuously ask us what we like, it then can infer and clarify through every day interactions. Now lets turn these  observations into practical, executable diagrams.

The Attention Architecture

In the last article, I talked about the architecture of the attention platform. This platform is designed to bring together attention capturing services, attention storage and attention applications that deliver end user value. The key aspects of the platform is decoupling between the services allowing various vendors deliver different implementations, yet communicate via common protocols.

                        
Let’s take a look now at how this platform can facilitate personalization applications such as  Idiomag

From attention to personalized news

As we discussed in the article last week, there are different kinds of attention – explicit and implicit. The implicit attention is simply the click stream with timestamps. The explicit attention are bookmarks or, in case depicted above, semantically rich bookmarks, captured by the blueorganizer. Note that in this example both attention sources are browser based.  This does not need to be the case. Attention information can come from your interactions with e-mail, desktop application and many other sources.

The job of the personalization engine, in this example it is the Touchstone engine, is to digest all attention information and produce the AttentionFilter. The filter is basically a discriminator that can assign a ranking to a set of documents. The documents that are highly relevant based on the user experiences get higher score than the ones that are not as relevant.

In case of personalized news, the documents are RSS feeds. The Idio service applies the AttentionFilter to the set of your RSS feeds and then ranks them. Based on this ranking it can either sort feeds for you or it can even filter things out. The resulting new set of articles is packaged in the form of a magazine and delivered to the end user.

Note the interesting feedback loop in this diagram. The magazine is shown in the web browser, and so the user attention will be again captured via explicit and implicit attention collection mechanism. This feedback loop will allow the user’s AttentionFilter to be refined and evolve as user’s attention evolves and changes.

Conclusion

We are not quite there in terms of the personalized news service discussed in this article. The blueorganizer and attention recorder are not storing information into the same Attention Vault. And Ideo today does not use Touchstone engine as a service. Instead it cuts through this diagram and asks the user to specify initial set of interests. It then evolves the preferences based on the implicit attention – articles that user clicks on. But this does not matter, because fundamentally, today, we already have examples of turning attention into personalization.

In the coming months we will be seeing more of these examples, and hopefully, we will together build robust attention platform that will fuel more and more interesting personalization services like Idio magazine.