Removing written reviews increases usefulness and authenticity of product star ratings

Removing the need for text-based reviews leads to more positive and useful product star ratings and decreases the impact of bought fake ratings, finds new research by Professor Matthias D. Mahlendorf from Frankfurt School of Finance & Management, Oliver Hegers from Vrije Universiteit Amsterdam, and Fan Wu from The Chinese University of Hong Kong.

In 2019, Amazon removed written reviews as a requirement when leaving a product rating, introducing one-tap ratings of one to five stars. Researchers investigated the impact of this policy change by following 500,000 products on Amazon week by week, 77 million observations in total, and used 50,000 books also rated on Goodreads as a comparison.

The researchers find that the previous written requirement had discouraged happier customers from leaving reviews. A year after the change, products tracked had accumulated over three times as many ratings as written reviews, and the average rating had risen from 4.17 to 4.32 stars. This indicates that customers previously put off by the written requirement, on average, were more satisfied than those who had taken the trouble to write.

Removing text reviews also makes fake ratings less effective. Using a detection method to identify products buying fake reviews, researchers found that these products had ratings one quarter of a star above comparable products before the change. Afterwards, the gap narrowed significantly. As more authentic customers leave ratings, each purchased five-star rating has less impact on the overall average, so buying ratings becomes a worse investment.

Ratings also become more useful. After the change, Amazon ratings lined up more closely with independent product tests, and sales were more strongly influenced by a product's rating. Reviews in which customers complained about having been misled by reviews declined from 36% to 21%.

“Our findings show that the way rating systems are designed can significantly influence which customers leave feedback and how accurately ratings reflect product quality. While Amazon’s decision to remove the written review requirement improved the informativeness of ratings in this case, design changes do not necessarily lead to better outcomes. Platform operators therefore play a crucial role in shaping the ratings and reviews consumers see and rely on when making purchasing decisions,” says Professor Matthias D. Mahlendorf from Frankfurt School.

The study, titled “Removing sludge: How rating system design affects the informativeness of crowd-based performance measures”, was published in the journal Accounting, Organizations and Society.

/ENDS

For more information, a copy of the research paper, or to speak with Professor Matthias D. Mahlendorf, contact Kyle Grizzell-Smith from BlueSky Education on +44 (0) 1582 790709 or [email protected]