Chat with us, powered by LiveChat

Online Reviews and Quality. Do They Even Correlate?

In college, I devoured a book called Zen and the Art of Motorcycle Maintenance.

Who didn’t?

I know I’m dating myself here. But, the book reappears on college campuses every decade. The book delves into what defines quality. The protagonist’s philosophical investigations drove him insane, and he was ultimately treated with electroconvulsive therapy, which permanently changed his personality. Still, the quest to define quality remains a worthy goal, if elusive.

The online world has become the sounding board for consumer’s perceptions of quality. A vote up or down on the Internet does correlate with whether a business will make more revenue – or less. Researchers studied the effect of Yelp ratings on revenue from restaurants in Washington. They concluded that each ratings star added on a Yelp review translated to a 5 percent to 9 percent effect on revenues.

What about healthcare?

We were early skeptics that online reputation correlated with quality in healthcare. Many years ago, we believed that online reviews were useful, at best, for addressing the soft, subjective factors related to the patient experience – for example, did the doctor communicate well, was his bedside manner adequate, how long did the patient have to wait for an appointment, etc.

We assumed that patient reviews were not helpful for addressing the more important factors of patient care – namely safety and clinical outcomes. It doesn’t matter if the doctor communicates well if his patients are dying because of negligent care.

So, we studied it.

One the data was in, we concluded we were wrong.

Our group was the first to survey and connect online reputation with patient safety and quality of care. Segal J, Sacopulos M, Sheets V, Thurston I, Brooks K, Puccia R. Online Doctor Reviews: Do They Track Surgeon Volume, a Proxy for Quality of Care? J Med Internet Res 2012;14(2):e50

So, we were the first to provide empiric data suggesting online reputation was helpful in guiding patients on metrics that mattered the most, proxies for quality of care.

Which brings us to a collection of provocative articles recently published in the Journal of Consumer Research. I’m positive that journal is on everyone’s must-read list.

The flagship article, Navigating by the Stars: Investigating the Actual and Perceived Validity of Online User Ratings, compared quality measures for over 1,000 consumer products. The head to head comparisons were based on Consumer Reports and Amazon reviews. In the former, experts tested the consumer products using sophisticated measures and tools unavailable to most consumers. In the latter, consumers weighed in with their subjective take on the products.

The authors concluded average user ratings from Amazon:

  • lacked convergence withConsumer Reports scores, the most commonly used measure of objective quality in the consumer behavior literature;
  • are often based on insufficient sample sizes;
  • do not predict resale prices in the used-product marketplace; and
  • are higher for more expensive products and premium brands, controlling forConsumer Reports

“…when forming quality inferences and purchase intentions, consumers heavily weight the average rating compared to other cues for quality like price and the number of ratings. They also fail to moderate their reliance on the average user rating as a function of sample size sufficiency. Consumers’ trust in the average user rating as a cue for objective quality appears to be based on an “illusion of validity.”

Other papers in the same issue pushed back on the findings. One group said, so what, what does it matter? Quality is in the mind of the beholder. If the consumer believes a product delivers quality, so be it.

That same group pushed back arguing that helping consumers achieve their goals with uncorrelated data may be useful. One analogy. If my goal is to preserve wealth and grow my savings, I want a well-diversified portfolio exposed to different risks. The trajectory of stocks and bonds do not often correlate. I want investments that do not correlate. So, having a basket of both gets closer to achieving my goal of mitigating risk and preserving wealth. If I had all my eggs in one basket, I might grow my savings more, but at the expense of more risk.

The original authors responded that objective quality does matter, and may even trump subjective sentiment. They provide two examples:

  • Beats Headphones. Apple purchased Beats, the high-end headphone company, in 2014for $3B. On Amazon, Beats has 4 star out of 5 star ratings with hundreds of reviews. Consumers seem to love the product. But, hardware engineer Avery Louie did a teardown analysis. He found that internal screws (which would add to production cost) were shunned in place of less expensive, and less durable snaps and plastic fasteners. More interestingly, Beats adds nonfunctional, heavy pieces of zinc to the headphones, potentially to give consumers the impression the construction is more solid than it really is. Consumer Reports ranks Beats as mediocre in quality and a bad deal at its premium price.
  • Baby seats. While consumers can provide feedback, what matters most to purchasers? That the seat will protect their infant if the car is in an accident. The number of reviewers who can comment on that from experience is limited. But, Consumer Reports does have sophisticated equipment to rank one seat from another.

So, there you have it. Now I know why the protagonist in Zen and the Art of Motorcycle Maintenance lost his mind deconstructing quality.

Still, here are some take home points:

  • If you are a physician, being defined by large number of online reviews makes it more likely you are fairly represented. You do not want to be defined solely by two noisy patients with a megaphone. The solution to pollution is dilution.
  • Online reputation does correlate with revenue – in healthcare that means new patient volume.
  • Online reviews are strong marketing signals; trusted by patients.
  • Online reviews validate a patient’s decision to follow-through on a referral from friend, family, or another doctor.
  • And our work still stands, for surgeons, online reputation correlates with proxies for objective quality of care.

In healthcare and other domains, online reputation does matter.




  • 2
By |2017-07-14T11:27:04+00:00May 20th, 2016|Marketing, News, Online reputation, Online reviews, SEO|2 Comments

About the Author:

Helping patients find the best doctors online. Helping the best doctors be found online.At eMerit®, we focus on managing your Dental or Medical Identity toward a sustainable and growing business – where both doctors and patients thrive.And we minimize distractions away from patient care while meeting your business objectives by transforming everyday patient interactions into growth drivers.At eMerit, we take your Medical Identity® personally.

Leave a Reply

2Comment threads
0Thread replies
Most reacted comment
Hottest comment thread
2Comment authors
newest oldest most voted
Notify of
Eric M. Joseph, MD

eMerit has been a welcome addition to our practice. Reviews are readily and easily composed by our patients, in our office after care is rendered, that has led to an online reputation that is true, balanced, and humbling. The negative reviews we all receive have more of an emotional impact on the physician, even though it is impossible to make everyone happy. The discontented minority that opine loudly and anonymously may cause insomnia from the sting – and we are limited by privacy laws with our responses. We highly recommend eMerit without reservation. I am happy to chat with any… Read more »

Interesting well constructed article. Yes, consumers do seem to worship at the feet of the “great Oz”…. also known as “online reviews”. Thanks for sharing.