Monday, October 09, 2006

Progress on Netflix recommendations contest

Seven days after the start of the Netflix contest, the first entry appears on the leaderboard that beats the performance of Netflix's recommender engine Cinematch.

See also my earlier post, "Netflix offers $1M prize for improved recs".

Update: John Chandler-Pepelnjak notes in the comments to this post that a new entry for a team called "The Thought Gang" just qualified for the "Progress Prize" of $50k. Excellent.

Update: Two weeks after the start of the Netflix contest, there are now six entries that beat Netflix Cinematch. The top entry, "NIPS Reject", has a nearly 2% improvement. Impressive.

Update: Things are heating up. Eighteen days after the start of the contest, there are now thirteen entries that beat Netflix and nine that qualify for the Progress Prize. The top entry has a nearly 5% improvement, halfway to the Grand Prize of $1M.

4 comments:

Anonymous said...

You should be all over this. With your experience at amazon and Findory wouldn't you be just about the most likely person to win this thing?

Greg Linden said...

I don't think so, no.

The problem posed by the Netflix contest is quite a bit different than what I normally do. Most recommender systems do what are called "Top N" recommendations, finding a small group of items that are likely to be of interest to someone. This Netflix contest requires a recommender system that can predict the expected rating of any item including items the customers will hate. That is a very different problem.

I am playing around with the Netflix data -- it is quite a bit of fun -- but I think this contest likely will be won by people with more substantial computing resources than I have available.

John Chandler-Pepelnjak said...

"The Thought Gang" appears to have passed the 1% mark on the leaderboard now. That didn't take too long.

burtonator said...

Wow.... this is pretty awesome I must say... I have to figure out how they determine the ranking improvement. Certain algorithms can work well under certain scenarios but the problem is the non-trivial edge cases.......

Anyway... pretty sweet. This is turning into the X-prize. Might actually be a model for tech people who aren't entrepreneurs.