The Price of Popularity · 2 June, 03:00 PM

It’s something I’ve talked about before: the holy grail. Recommendations in an aggregator seem like a good idea. Somehow it just never works out that way. I’ve come to think that this is one of those features that people would love to have, but the actual implementations fall far short of what people consider useful. I’ve looked at I don’t know how many of them: Bloglines’ recommendations, Amphetarate, NewsGator’s implementation, Findory, who knows what else. All of them suffer from the same problems: they require too much input, the output’s in the wrong format (for example, recommending feeds instead of content), and mostly, the results just aren’t interesting.

One of the things I mentioned in my post I linked above was that when I was working on recommendations, what I found was that the really interesting stuff came in the 2nd quintile of recommendations my algorithm generated. The final form I produced did exactly that: generated a list of recommendations and cut off the top 20% to cull the obvious recommendations off of the list. I think that one of the key elements of a recommendation algorithm is the idea of surprise; if recommendations produce obvious candidates, they won’t remain interesting to a user. Giles Bowkett arrived at roughly the same conclusion “Even if you didn’t know about the long tail, you’d look for the best ideas on Hacker News (for example) not in its top 10 but in its bottom 1000”

The best recommendation system I have is my aggregator itself: I would never have read Giles’ post if it weren’t for Dare and Greg Linden linking to him. We work this way all the time: we take recommendations from friends, neighbors, random people on the Internet… :) But this is imperfect. I was talking movies with neighbors the other day; we normally share the same taste in movies, but when we got to talking about No Country For Old Men, we disagreed sharply. This was a movie I liked – not great, but interesting, yet my friends turned it off halfway through.

I always held out a lot of hope for recommendation systems, but I think I’ve lost my faith in the idea, barring some radical breakthrough.

— Gordon Weakliem

---

Comment

  1. Something in a similar vein that I’ve seen popping up is the idea of recommending something based on others with similar taste. For example, Amazon shows books that others who purchased this book also bought. Many music sites suggest similar music, either based on complex mathematical analysis or on the frequency of it being in a playlist with the current song. Your recommendation could compare the user’s subscriptions list (and maybe weigh it by frequency of activity, in case they let some feeds just pile up without being read) with a database of anonymous-but-updated subscription lists. If 70% of the people who subscribe to your blog also subscribe to Greg Linden, then tell them that as a recommendation. Allow them to explore the correlations, even down to the “long tail”, to see that single coincidental interest in basket weaving, French cuisine, or radio astronomy.

    — J-P Losier · Jun 24, 09:51 AM · #

  2. Sure, but what if your sample set is statistically insignificant? A according to FeedBurner, I have about 400 subscribers – what if the number of people who subscribe to both me and any other particular subscription is very small?

    — Gordon Weakliem · Jun 26, 07:38 AM · #

Commenting is closed for this article.