Netflix to Replace Star Ratings With Thumbs Up/Down
Tim Hardwick (via Luc Vandal, Hacker News):
Previous star ratings given by users will be used to personalize their Netflix profiles, but the ability to rate a TV series or movie by awarding stars is set to disappear altogether, according to Variety.
[…]
According to Netflix, at one point subscribers had awarded over 10 billion 5-star ratings and more than half of all members had rated more than 50 titles. However, the company eventually concluded that star ratings had become less relevant, with some users giving documentaries 5 stars and silly movies just 3 stars, even though they would watch the silly movies more often than the highly rated documentaries.
“We are addicted to the methodology of A/B testing,” Yellin said. The result was that thumbs got 200% more ratings than the traditional star-rating feature.
I don’t think people agree on what a rating is supposed to mean, anyway. Secondly, the thumbs aren’t really a binary system because you can choose not to rate at all. So it seems to me that they’re effectively just getting rid of the 2- and 4-star options.
Update (2017-03-19): Jeff Johnson:
I think movies should be rated by how many times you’d watch them. 0 if you regret watching, 1 for a decent movie, and going up to infinity.
Update (2017-03-20): John Gruber:
For a personally curated collection, 5-star ratings can be meaningful. But for a recommendation service that averages ratings among all users, they are not. It’s the difference between designing for the ideal case of how people should behave versus designing for the practical case of how people actually behave.
Binary ratings make a lot more sense in certain contexts, and with YouTube, it’s a natural fit. You don’t rate a movie on YouTube; you generally rate a cat video, a TED Talk, or something short.
I disagree that this type of rating will work on Netflix.
Update (2017-04-27): Jason Snell:
So in an act of defiance against the old order, the web ended up embracing one of the old order’s failings—the arbitrary five- or ten-point scale for rating things. Now you, too, can invent your own personal code for the films you see or the products you buy. The problem is that there’s no one right way to make a ratings system. At Macworld, we had a rubric for reviewers to follow that instructed them what all the ratings levels meant, because we wanted our ratings to be consistent. I could gauge the sentiment of a review by reading it, and equate that to a number.
[…]
Siskel and Ebert had it right. The two critics were forced to provide star ratings for their newspapers, but when they created their own TV movie-reviews show, they famously boiled the whole thing down to thumbs up and thumbs down. And they were critics who reviewed hundreds of films a year! If it was good enough for them, it’s good enough for the rest of us—and for the algorithms fed by our sentiment.
