Tuesday, January 26, 2010

Sort by Rating

A plea to Overstock (as well as any other web retailer which is willing to listen):

When I view a page of items from a category, I can select various "Sort By" options, such as "Top Sellers" and "Review Ratings". When I'm not looking for something specific, I would like to know what items are both popular and highly regarded by other customers. Neither "Top Sellers" nor "Review Ratings" does this. "Top Sellers" ignores ratings and a product with a single 5-star rating can be the top product according to "Review Ratings".

I would like to suggest an improvement to the "Review Ratings" sort which would make it more useful. Currently, you sort by average rating, i.e. sum-of-ratings divided-by number-of-rating. If you add a fictitious 3-star rating to this calculation (sum-of-ratings-plus-3 divided-by number-of-ratings-plus-1) the sort will be more valuable as only products which are popular AND receive top reviews would make it to the top of the list.

'course, anyone with a hint of machine learning background will recognize this as a very basic Bayesian posterior estimate. IMDB is one of the few sensible web sites which uses this sort of estimate for sorting based on rating.

Update (3/5/10): As Ashwin noted in the comments, the Wilson Score may be an even better solution.


  1. This is probably what you are talking about :) - http://www.evanmiller.org/how-not-to-sort-by-average-rating.html

  2. Coming late to the game on this one, but the Wilson score is not good because all low-vote products start out at the bottom of the list. I explain why and give a basic intro to an example multinomial/Dirichlet posterior computation here: