Yes, in fact, there is accounting for taste

Sometimes you find a video-store clerk who knows exactly what movie you'll like. Or a clerk in a music store who senses your taste in bands. Or a bookworm who can deliver one terrific novel after another from the shelves.

Marc Pickett wants to take luck out of that equation. And win a million dollars in the process.


Pickett, a doctoral student at the University of Maryland, Baltimore County, is trying to perfect a "recommender." That's a computer program designed to analyze your cinematic tastes and predict what movies you'll like.

Online retailers, including Netflix, and Apple's iTunes music store, put great stock in recommenders. They're critical tools in their efforts to establish long-term relationships with customers and sell more products - such as movies, books, album tracks and other goods.


The program is particularly critical for Netflix, an online mail-order movie-rental giant whose livelihood depends on keeping customers happy enough to pay $5.99 or more every month for the opportunity to watch its videos.

In October, the company offered a $1 million prize to anyone who could develop a program 10 percent more accurate than its current recommender, known as Cinematch. A chance at that chunk of change set thousands of programmers around the country, including Pickett, to work on the problem.

"Imagine that our Web site was a brick-and-mortar store," said James Bennett, Netflix's vice president of recommendation (yes, that's his title). "When people walk through the door, they see the DVDs rearrange themselves. The movies that might interest them fly onto the shelves and all the rest go to the back room."

Elsewhere, work on recommenders has led to prototypes of mood-sensing digital music players and a program that produces virtual, three-dimensional maps of customers' music collections.

Bennett said the company hopes the contest will ultimately enable the Netflix site to offer even more personalized lists of movies to customers.

But improving algorithms that dispense advice is no trivial task, according to computer scientists and mathematicians working on the problem.

One hurdle is simply having enough raw computing power to study people's likes and dislikes in the first place.

To make its predictions, Netflix's Cinematch churns through a billion movie ratings the company has collected from its customers over the years. The program clusters customers in groups based on how they rate movies on a five-star scale.


"The problem is to help you find soul mates," said John Riedl, a professor at the University of Minnesota who developed an early recommender in the 1990s. "It looks for people who felt the same way you did about some movies and makes suggestions based on what other movies they liked."

A fan of The Godfather, for example, might like Goodfellas, Raging Bull, Scarface, Taxi Driver and Platoon, based on the viewing preferences of other fans of the classic 1972 gangster epic.

At Netflix, Bennett said, Cinematch can predict a customer's opinion of a movie within a half a star, on average. To improve on that score, the company has released anonymous data on 100 million customer ratings so that Netflix Prize contestants can test their algorithms on real people.

"We're using four souped-up computers, but we can only feed bits of the data in at a time," said UMBC's Pickett, standing in an office cluttered with computer monitors, keyboards and robotic parts.

His computer screen displayed a map he created to visualize the movie preferences of thousands of customers. People who appear in proximity on the map, Pickett said, had similar taste in movies in the past and thus are likely to feel the same about movies in the future.

The UMBC team hopes to submit the results of its first trial by the end of the year. "Winning the $1 million would be nice," Pickett said, "but we're really after the bigger prize: artificial intelligence."


As of Thursday, the frontrunner among the 13,487 teams competing for the prize had improved on the accuracy of the Netflix algorithm by 6.11 percent.

But no matter how good recommenders get, the vicissitudes of mood and taste make it unlikely that these programs will ever approach perfection, experts said.

Consider the classic 1950 science fiction movie Plan 9 from Outer Space, by director Ed Wood.

"Some people consider it the worst movie ever made, so most sci-fi fans won't like it," Pickett said. "But film buffs might watch it to figure out what makes a movie really bad."

Matthew R. Kuhlke, co-founder of the movie Web site, said he knew something was wrong when his software suggested his mother rent the Adam Sandler movie Billy Madison. She wasn't amused by Sandler's brand of juvenile humor.

"It's one of my favorite movies, but she hated it," recalled Kuhlke, who lives in San Francisco. "I realized we needed to do some fine-tuning."


WhattoRent has a different take on mind reading than the "wisdom of the masses" approach used by Netflix, Amazon and iTunes. It categorizes movies according to criteria such as how many big-name stars appear, whether it's depressing or uplifting, and how predictable the outcome is based on early scenes.

It analyzes visitors' movie ratings against these criteria to connect them things they might like., an online radio station based in Oakland, Calif., uses a similar approach to pick music. Pandora staff members analyze about 10,000 songs a month for 400 musical qualities such as "subtle use of vocal harmony," "pop rock qualities" and "major key tonality."

"We've broken the songs down into musical primary colors," said Timothy Westergren, the company's founder. "All of those things form a kind of musical DNA."

When listeners type the name of a song or artist into a form on Pandora's Web site, it searches through a database of 500,000 titles to create a playlist of songs that share its qualities. Listeners can rate or skip songs as they play through the Web site. This provides feedback Pandora uses to decide what to play in the future, Westergren said.

One drawback of these "content-based" systems is they don't always capture musical elements important to some customers, said Paul Lamere, a computer engineer at Sun Microsystems.


"If the main reason you like music is how much cowbell is in a song, but the system ignores cowbell, you'll never get a good recommendation," he said.

But Lamere still thinks the approach has merit and is taking it one step further with his own program, Search Inside the Music.

Instead of having humans analyze songs for their constituent qualities, Lamere's computer program does the job automatically. The software analyzes the digital music and words of a song, then categorizes it according to 10 criteria, including lyrics, musical theme, melody, tempo, rhythm and instrumentation.

With this information, the program generates a three-dimensional map of a music collection in which similar songs are clustered together. On the computer screen, a listener can explore the collection by "flying" through it as if playing a video game."

Search Inside the Music can also take listeners on a musical journey, Lamere said, transitioning over time from playing one musical style to another, much like an experienced disc jockey.

"I can tell it to play upbeat music as I leave work in my car," Lamere said, "but transition to calm, relaxing music by the time I get home to my wife and kids."


Too busy to tell a recommender what you'd like to hear? Sandor Dornbush, another UMBC graduate engineering student, is working on a mood-sensing MP3 player to free you from that task.

The portable player, for now dubbed the XPod, will monitor physiological signs such as heartbeat and skin temperature to determine what kind of music to play, Dornbush said.

"It could find, for example, that when you run you like upbeat music," he said.

But like all recommenders, human or digital, Dornbush's prototype needs to spend time with a person to get to know him.

"If you don't give them any feedback," Dornbush said, "they have a tough time figuring out what you like."


Need advice?

Do you need help finding music, books, movies or something else? These Web sites offer advice using programs known as "recommenders." Just don't expect miracles.







Web sites and blogs