Ratings and Reviews

Definition

A rating is a score a customer gives, usually one to five stars; a review is the written text that may come with it. Rating counts are usually larger than review counts, and the average is shaped by who chooses to rate, so it measures the satisfaction of raters, not of all buyers.

In plain terms

Many marketplaces count ratings and reviews separately, and some people give stars without writing anything. The two numbers are not interchangeable, and neither is a sales figure.

Star ratings tend to be J-shaped: many fives, some ones, few in between, because very happy and very unhappy customers are the most likely to rate. An average of 4.4 hides that shape. An average also needs a count to be read: 4.6 from 12 ratings is much less certain than 4.4 from 3,400.

Example

Two synthetic products, ratings with a standard deviation of about 0.9 stars:

P’s higher average says almost nothing, because 12 ratings could easily produce it by chance; Q’s lower one is reliable. Ranking by the bare average puts P above Q.

Common misunderstandings

• “More stars means a better product.” It means raters were more satisfied; who rates, and how many, shapes the number.
• “The rating count is the sales count.” Only a fraction of buyers rate, and the fraction varies by product and platform.
• “Reviews are a representative sample of customers.” Reviewers are self-selected.