This is what Amazon Reviews Should Look Like
As the founder of ReviewMeta.com, I’m always asked what Amazon could do to improve things. I’ve spent a great deal of time thinking about this, and my answer involves 3 steps:
1. Quick & Easy: Only Allow Verified Purchase Reviews
Amazon claims: ‘Reviews that are not marked “Amazon Verified Purchase” are valuable as well‘. However, I agree with the majority of feedback I receive from shoppers that there’s no reason to allow unverified purchase reviews. Period.
Our data suggests that Unverified reviews are less reliable than their Verified counterparts. Looking at reviews on Amazon.com in 2017, unverified reviews were more than twice as likely to end up deleted as their Verified Purchaser counterparts.
If Amazon is deleting so many Unverified reviews, why are they still allowed on the platform? I believe that Unverified Purchase reviews are doing more harm than good.
2. Vastly Increase Transparency
At ReviewMeta, my belief is that more information is always better. Amazon already discloses a few pieces of pertinent info, but you are often required to click to a profile page to get there. Here’s a mockup of how I believe Amazon’s review page could look:
At first glance, it looks like a lot of extra info, but I believe each piece of information is important in helping shoppers understand the background of each review.
Here’s a few pieces of information that should be summarized and displayed for every product:
- Show suppressed (deleted) reviews: At ReviewMeta, we can see that millions of reviews have been deleted on Amazon. In our opinion, this is a good thing – Amazon is taking action on the fight against fakes. However, I think that customers should be aware when something like this is taking place.
- Monthly rating trend: This is something already available through ReviewMeta, but it’s valuable for all shoppers to see. Has the average rating been going up or down in recent months? With 1,000’s of reviews, it might be hard to tally up yourself, so a summarized graph would be extremely helpful.
For every review, show:
- Price paid: Shoppers will be able to tell if someone picked up the product at a steep discount or overpaid. It can also raise red flags if all the reviewers seem to have paid a substantially higher or lower price.
- Seller purchased from: There have been many stories about people receiving totally different products from the same listing on Amazon. Sometimes a seller will offer a lower price but ship a knock-off version of the product, resulting in a flood of negative reviews for that product. Being able to tie each review to a seller can help shoppers avoid counterfeit products.
- Purchased, delivered, reviewed, returned and edited dates: Currently, Amazon only shows one date on the review. Shoppers don’t know if that reviewer left feedback 15 minutes after receiving the product or 15 months after receiving the product.
- Average rating from reviewer: While this info can technically be tallied up (and is already displayed on ReviewMeta), I think it should be more readily available on every customer review on Amazon. This helps shoppers get a quick sense of how critical a reviewer is.
- Number of Reviews/Total Helpful Votes: This is already available on the profile pages, but it would be easier if it was also displayed alongside the review.
- Number of items purchased from Amazon/Prime Status: Here’s where we could run into some privacy issues, but if Amazon could pull it off, it would be a significant win for transparency. Would you rather trust a review from someone who has bought a single item from Amazon or a prime member who is making consistent purchases?
- Brand reviews/rating: ReviewMeta is already looking for Brand Repeaters, however displaying it next to every review on Amazon would be another amazing display of transparency.
The added transparency would totally transform the review platform and enable shoppers to get a better sense of where the reviews are coming from and which ones to trust.
3. The Ultimate Goal: Trash Amazon Vine, Replace with Amazon Testers
In 2016, when Amazon banned incentivized reviews, their announcement included a promise about improving the Vine program:
We also have ideas for how to continue to make Vine an even more useful program going forward. Details on that as we have them.
Nearly two years later, I still haven’t seen any improvements in the Vine program, let alone any mention of planned changes to the Vine program from Amazon.
Related video: What is the Amazon Vine program?
I think that the Vine program has potential, but Amazon doesn’t seem to be focusing their energy on improving it. The biggest problem seems to be the extremely high (and not very clear) cost to sellers. Personally, I still rarely see Vine reviews when shopping on Amazon, and have yet to see a single “Early Reviewer”.
Proposed Solution: The Amazon Tester Program
Shoppers don’t need tens of thousands of anonymous customer reviews to help them make a decision. Even a hundred Vine reviews only gets you so far.
Shoppers just need a few high-quality, thorough reviews from a trusted source – the Amazon Testers.
Here’s how it would work: Amazon would hire a team of “Product Testers” to work for them. Yes, they’d be on the payroll and actually go into the office to test products and write reviews all day.
Product Testers would have access to a studio and high-end camera equipment to quickly and efficiently shoot videos for all their reviews. They’d also have access to other scientific testing equipment to compare different types of products. Think “Mythbusters” but for gadgets on Amazon.
Rather than lumping these reviews in with the rest of the customer reviews, you’d see them in their own special section with their own average rating. Yes, that means products could have more than one rating – a Customer Rating and a Tester Rating.
Trust, Experience and Quality over Quantity
Just one or two reviews from a trusted, experienced source can be much easier for a shopper to digest than tens of thousands of customer reviews. Here’s why:
- Specialization: Testers would be specialized in reviewing products from one or two categories, for example Audio/Electronics, Kitchen Appliances, Bath and Beauty, etc.
- Experience: Testers would have experience with hundreds of similar products and be able to sort out the good from the bad. Would you trust a review from someone who just bought their first pair of headphones or from someone who has professionally reviewed hundreds of pairs?
- Resources: Ideally, Testers would have access to scientific testing equipment to factually determine if products live up to their claims. No more “guessing” if something works or not.
- Trust: Knowing that these testers are employees of Amazon and not anonymous “customers”, shoppers won’t have to worry about them being paid shills for the brand.
- Apples-to-apples comparisons: Competing products will likely be reviewed by the same handful of testers. With Tester Reviews, brands won’t be able to selectively solicit reviews from the easiest graders.
The whole program would eventually pay for itself.
The most obvious concern here would be the enormous cost and manpower required to test the millions of products available on Amazon. If Amazon implemented this correctly, the entire Tester department would end up paying dividends.
First, Amazon would only be testing the most popular products. There’s a lot of ways to measure product popularity, but I would look at sales.
So, for example, Amazon spends $500 testing a pair of headphones that has already generated $500,000 in sales. That is one-tenth of one percent of total sales to get a solid review on a really popular product. If that expense was transferred directly to the consumer, the pair of headphones that was originally $9.98 is now going to cost you $9.99.
Second, Amazon Testers could help increase consumer confidence and ultimately generate more sales. 10’s of thousands of Amazon shoppers use ReviewMeta daily because they are skeptical of the existing customer review system. The Amazon Tester program would help address this lack of trust and keep more shoppers on their platform.
Third, the content generated from Amazon Testers would ultimately drive traffic directly to Amazon. Shoppers often do research outside of Amazon customer reviews to help them make purchase decisions. Having ultra-high quality reviews from experienced product testers will draw in more shoppers looking for in-depth product info – directly from Google and other search engines.
For example, I was recently looking for a new high-end cooler. Customer reviews for all potential products were mixed, but all I wanted to know was which had the best insulation. I had to leave Amazon to find someone who had tested all the top coolers and had the results available.
In the end, nothing is perfect and it’s easier said than done.
As an experienced web developer, I know that just because someone comes up with an awesome idea doesn’t mean that it’s easy (or possible) to implement. It’s easy for an outsider to sit here and say “Amazon should just block unverified purchase reviews” or “They should simply show the purchase and delivery dates for every review”, but it might not be that easy.
What if there are some legal reasons behind allowing unverified purchase reviews? Or maybe they don’t actually have the data to retroactively go back and show the purchase/delivered dates for all the hundreds of millions of existing reviews. There are a lot of technical challenges that I’m completely unaware of.
Without having a behind-the-scenes look at how Amazon works on the inside, I have no idea how practical or possible some of these solutions would be. However, in a perfect world, this would be my starting point.
There’s ultimately no 100% perfect solution to completely remove all bias from product reviews. Shoppers should always stay critical of what they read and thoroughly research products before making a purchase. Increasing transparency on the customer reviews and hiring professional testers simply give shoppers additional tools and resources to fully conduct their research and ultimately make their own decision.
Amazon Testers could never come close to providing sufficient value to justify its cost – just consider the scope of the problem space, spanning wireless electronics, heavy machinery and tools, lingerie, and plant seeds (just to name a few). Plus, there are just so many ways for it to backfire, from legal liabilities to accusations of bias or seller manipulation (i.e. testers get something different from consumers) to just plain getting it wrong. And even with the very best of intentions, I’ll never value professional reviews half as much as a good sampling of legitimate consumer reviews based on real-world experience. Such reviews are already easy to find (when they exist) based on the quality and specificity of their more-than-three-words review content. Forget professional reviews – I would get more additional value just from more prominently featuring the “top critical” reviews rather than making me navigate around for them.
Exposing purchase dates of reviewers is a considerable privacy invasion for value that is questionable at best. At the very most, the only number that should be shared is the number of days between delivery date and review submission (generalized to orders of magnitude, i.e. owned product for x days, weeks, or months).
Showing the price paid is also problematic, because it is so manipulated by shipping costs, like products that sort to the top for ridiculously low prices but just moved all the cost into “shipping,” or conversely products with price inflated to include shipping cost.
In your rush to push transparency (not a bad thing in itself) I think you’re forgetting that every piece of data newly or more prominently shared is also a new vector for manipulation. In the best-case scenario scammers will adjust to the new evaluation metrics and potentially just make it harder for discerning researcher consumers to reliably get the facts already available.
Good points. The professional reviewers would only be reviewing the most popular products. So Amazon would only invest in getting reviews for products that are actually doing a lot of volume. They wouldn’t be reviewing every last product sold on Amazon as it would be astronomically expensive.
As far as the “top critical review”, Amazon already shows this and it’s open to manipulation. You’ll see a brand post a 3-star review that’s actually quite positive (eg. “It was a really great product but didn’t quite fit the use it’s not even designed for”) and then have a bunch of their employees click “helpful” on that review and boost it to the “top critical review” spot.
I think showing the “length of ownership” is much better and more user-friendly than the purchase/ship/receive dates. The length of ownership is really what I’m looking for when reading a lot of reviews. I want to know how this thing holds up 2 years after I buy it, not 2 days. Also, I figure a lot of the fake reviews would all be posted one day after purchasing…
The price paid I can see being a challenge, but it might be solved if it was shipped free through prime. Or maybe just show the total price paid.
Whenever I have purchased a product from or through Amazon, I often look at the customer reviews. However, in my opinion, all reviews of this kind should be taken with a huge grain of salt even the ones where the person is at least trying to post an honest forthright review. I may get a bit long winded here so please bear with me. I have on a number of occasions been invited as a verified purchaser to review an item. At all times, I have used the product for a reasonable period of time and have done my best to write a fair and honest appraisal of said product. With that being said, even my own reviews should not be taken as the last word on a product. What aggravates me about Amazon customer reviews is that so many of them are poorly spelled grammatically incorrect diatribes against a product replete with exclamation points and may often have little to do with being an actual review. This is why I consider most such reviews to be essentially useless as a means of deciding whether to buy something. If possible, I try to find a professional review on a product in which I am interested. As for Review Meta, I appreciate the effort that you are making. Yet I often find that whatever adjustments, that you make on the reviews of an item often do not appreciably change the overall rating of a product in terms of the number of stars, etc. I consider your site to be just one step in my decision-making process.
Yes, definitely two different schools of thought here. You’ve got valid points as well. Great to have a discussion involving all angles of the story.
OMG! Yes! This! I believe that unverified reviews should be accepted on books, though, because of programs like Prime Reading or Kindle Unlimited. Those reviews end up showing up as unverified (or at least they did the last I looked). OR Amazon should change those to “verified” because they can track whether or not you download a book. BTW, you just saved me from a $50 purchasing mistake and I shared your tool on my Facebook wall. Additionally, I contacted Amazon about the product reviews, armed with your information. One product with 252 reviews – and not a single one of them verified and all of them 5 stars – and it wasn’t a book.
Thanks for the share! Wow, 252 reviews, all 5 star and none verified? I just don’t understand how Amazon lets that happen…
I did report it. I got a reply that they couldn’t let me know what, if any, action they would take. I don’t know how Amazon allows these things either.
In fact, I didn’t even realize that none of the reviews were verified until I ran it through Meta. Meta said it was probably fake because it had NO verified reviews. I had to double check to make sure your algorithm didn’t miss anything. Nope. You caught it.
I think this is a duplicate. I responded to the other reply.
Only verified reviews for most products, maybe, but an absolutely terrible idea for book reviews. Most of my readers borrow my books, and your policy would prevent them from having a voice. Similarly, library book borrows are a source of reviews. I quite frequently review books on Amazon that I’ve borrowed from the local library.
I think that for some authors this makes sense, but the unverified purchase reviews are still an avenue for lots of abuse. Look at what happened a few years back to Amy Schumer: https://reviewmeta.com/blog/the-anti-amy-schumer-amazon-brigade-a-breakdown/ – most of the nasty reviews were unverified purchases.
Same things happened to a lot of highly polarizing books in the 2016 election and even after. Most of the problematic reviews were unverified purchases.
Honestly, this post is really focused on consumer electronics and other gizmos and consumables. I see book reviews (and movie reviews, etc) as a completely different animal that would require a totally different platform to help ensure authenticity. Possibly another blog post.
Fair enough. I think the logic is strong for miscellaneous “gizmos.”
Good starting point for sure. Let’s hope Amazon is already working on some of this.
Worth mentioning in your blog is to encourage the use of the “Not Helpful” button in every review, if warranted. If more people voice their opinion of the lack of quality of a review by the use of that button, perhaps Amazon will see the need for change, sooner.
Keep up the good work.
Absolutely! I’ve got a post I’m working on about “how to fight back”!
Hmm…the “not helpful” button has been removed from Amazon for some time. They left only the “helpful” button. This is true of reviews and Q&A’s. Besides, Amazon tracks patterns in the use of this function per user and stops counting them when used too much by the same user from the same IP address. They are much more sophisticated and aware that they are given credit for.
Yes, there are plenty of things you can do to improve your reviews, but using the helpful button is not sustainable.
I still have both the helpful and “not helpful” buttons. I didn’t realize they were going to do away with them! I use the “Report Abuse” link when I see reviews like what happened with Amy Shumer. I figure a report of abuse is more likely to get Amazon’s attention than an “uphelpful” upvote, but I click that, too.
There are some great thoughts here. Honestly, for the research-heavy consumer, these changes have some benefits. However, I’d guess that for the larger percentage of shoppers who don’t care to invest as much effort into each and every purchase, this is over-kill.
Take the increase in transparency in your section #2. Price is not something that Amazon will start tagging on reviews. Why? Because it will easily destroy satisfaction among customers and reduce sales. With every review that customers read with a lower price they will feel they should not buy at that time. Those that come on to review and see existing reviews at a lower price will rate products lower on a comparative price dis-satisfaction. Additionally, the sharing of so much detail about the order and the reviewer lowers anonymity which in return lowers participation. I’ve seen review rates between 1-2% now, why lower them from there?
When reviews were first introduced on Amazon in 1995, it was so simplistic. Customers rate a product and write about their experience. The core purpose of reviews is to validate a purchase decision.
No, reviews can’t revert back to this simplistic framework, but I’d argue that a divergence from simplicity will increase confusion among the less tech-savvy, create unintended negative side-effects, and lower participation rates.
Don’t complicate things, just keep the consumer-facing display of reviews as simplistic as humanly possible. What about protecting authenticity? That should be done behind the scenes with algorithmic prevention of review posting based on user and listing review patterns.
If you’ve been following subtle changes Amazon (meaning they were never announced, and this is by design) has made over the last several years, their changes back this approach. I’ll give you a couple examples:
1. Amazon limits the view of actual review text:
This was done by moving the “see all reviews” button to the very bottom of most listing pages in a very small font. This puts more focus on the star rating, the total review count, and gives a snippet of text through just the top couple most helpful reviews.
2. Removing performance metrics from reviewer profiles:
Amazon removed the “helpfulness percentage” by reviewers. They also allow reviewers to hide review content from their profile. This reduces influence of any given reviewer and puts the influence back on individual reviews as individual “votes” on individual listings.
3. Removal of review comment thread alerts:
This basically turned of Amazon customer conversations happening on individual reviews because users couldn’t know when a conversation was happening.
These and other changes are being done to limit the human element within review interactions on site. I’d guess this is because Amazon knows the number side of reviews converts sales more than the text side. So, focus on the numbers and limit the text. That’s what they are doing. I think this is fine, assuming the authenticity algorithm is functioning properly.
I just hope this post wasn’t intended to be some self-aggrandizing call to Amazon that they should adopt all your ideas. We all know that the timing your YouTube video about star averages preceded the Amazon 10/3/2016 banning of incentivized reviews, but it would be naive to think it was the cause for the change. (I’m not claiming that is the case here.) Amazon is such a big company that things move slow. That decision was likely made long before this video:
https://www.youtube.com/watch?v=kdLI62JKpCk&t=72s
I definitely see your points about what Amazon is doing, but it seems these are all things done to help drive sales, not necessarily help inform and educate the consumer.
“So, focus on the numbers and limit the text. That’s what they are doing. I think this is fine, assuming the authenticity algorithm is functioning properly.” – I think the authenticity algorithm is either not functioning correctly, or the consumers are losing faith in it at an increasing rate, or both. Consumers are getting more and more skeptical of online reviews, and I believe that more transparency is the next logical step to help regain confidence – not less.
“I just hope this post wasn’t intended to be some self-aggrandizing call to Amazon that they should adopt all your ideas.” – Not the case here. As the founder of ReviewMeta.com, I’m always asked what Amazon could do to improve things. This blog is used as a way to help help address common questions and topics in great detail.
I agree that we should consider the consumers’ needs first, but have you ever thought that sales numbers are a success metric of what consumers want? People vote every day on brands with their dollars, just like they do with stars.
Review Meta brings very valuable insights to the table, but it’s a simple fact that Amazon has been around for a lot longer, has more resources, and more data to make decisions. (What Review Meta has done in the review space is impressive, given the comparison to such a giant.)
I work for a very large seller on Amazon and have closely monitored even the smallest changes on the platform for 4 years now. I’d say they have and are making adjustments to review displays that optimize what is good for consumers. When consumers buy products, that IS consumers saying the purchase process was helpful to them. Most consumers will never want to invest so much time in their review research that they want to know the entire purchase history and background of every review. They just want a 3rd party confirmation that a product is good- that’s all.
Here’s my biggest point:
Everything you’ve outlined is important info to be considered when discussing review authenticity, but it’s simply overwhelming for the average end consumer. It should NOT be displayed on the front end. Amazon likely already considers most of what you’ve outlined, but it’s on the back-end as part of their authenticity algorithm. (One example is the difference between “tester” star ratings and customers star ratings. Amazon already does this with Vine on the back end for program participants.) By considering these variables of authenticity without displaying everything on the front end, Amazon protects the consumer from an information overload while maintaining as much integrity as possible. Putting all the updates you’ve suggested on the front end shifts the responsibility of determining authenticity more to the consumer. This is a burden on the consumer. (That doesn’t mean I think the current system is perfect. Amazon should still strive for improvement…but let’s continue.)
If you don’t believe me when I say most consumer don’t want this info, just look at these numbers:
Number of Review Meta plugin Users: 43,061 (Per Chrome store as of 9/12/18)
Number of Amazon Prime Members: 100+ million (Per Amazon’s 2018 shareholder letter)
As a general statement, that means only 0.04% (4 in 10,000) of Prime members use Review Meta. With this said, I’m a very avid user in that minority. I’m all about improving review reliability, but the fact is, the masses don’t care to see all the variables surrounding a review’s authenticity. They just want a reliable star rating and a buy box. Don’t overload the 99.96% to make the rest of customers ecstatic.
I strongly disagree that sales numbers are a metric of what consumers want. It’s what Amazon and their shareholders want. People vote on brands and products, but they may have been unfairly tricked or persuaded to do so.
I see how this could initially be seen as data overload to the average Joe, but they don’t necessarily have to look at every last detail. Also, I think some of these can be shifted to “flags” instead of just pure data. For example, instead of showing the review count and average rating for each user, only show a “flag” if that user has posted one review, or if they give mostly 5 stars. Or a different flag if they reviewed the product on the same day that it was received, or if they paid a substantially different price for the product.
There’s two schools of thought here – either Amazon should “figure it out” and basically tell the consumer which reviews to trust (I know I’m oversimplifying) or that Amazon should show us the cards and let the shoppers “figure it out”.
I absolutely see how the first school of thought is much simpler for the average shopper who just need to get a coloring book for their daughter and doesn’t want to spend 45 minutes conducting a review analysis. But I think the important point is that they don’t have to. Anytime someone purchases an Echo, do you think they sit down and read all 100,000+ reviews that are available? Probably not.
These are very good ideas! I hope Amazon sees this!