Browser Extensions 

Unverified Review Problem

Amazon Flooded with Millions of Fake Reviews in 2019

March 28th, 2019

Key findings:

  • Amazon has seen a massive spike in Unverified reviews (reviews without the “Verified Purchaser” badge) in the last few months.  
  • Of the 1.8 million Unverified reviews posted in March 2019, 99.6% are 5-star.  In comparison, during 2017-2018, we saw an average of ~300k unverified reviews per month, only 75% of which were 5-star.
  • These reviews are mostly posted for cheap, off-brand electronic products, and seem to be pushing them onto the first page for popular searches like “iPhone charger”, “bluetooth headphones”, etc.

I’ve honestly never seen it this bad.  It all started when I noticed an increase in complaints on Twitter about cheap electronic products on Amazon that were flooded with blatantly-fake 5-star reviews.  They all seem to follow the same pattern:

  1. Off-brand, cheap electronic item (headphones, chargers, cables, etc)
  2. Literally 1,000’s of reviews
  3. Nearly 100% unverified purchases
  4. Nearly 100% 5-star reviews
  5. All reviews created over just a few days
  6. Reviews for dozens of different random “variations”, such as “Color: MC0326-C2”, “Color: Blue-42”, etc.

Screen Shot 2019-03-28 at 8.56.36 AM

Screen Shot 2019-03-28 at 8.58.22 AM

It’s not like these products were hidden deep in the depths of Amazon’s catalogue – this fake review scheme seems to be pushing these products to the first page of search results. We recently posted an article showing that 10 of 22 results for the search term “iPhone Charger” were products that had this same pattern of fake reviews.

At the time of writing, this off-brand MacBook Pro Charger has the “Amazon’s Choice” badge, even though the product has thousands of blatantly fake reviews.  

Screen Shot 2019-03-27 at 5.44.28 PM

Our analysis of this product throws out every single one of the 2,855 reviews.

Screen Shot 2019-03-27 at 5.46.26 PM

If you’ve spent any time on ReviewMeta, you’ll know that we are very reluctant to use the word “fake”.  However these reviews are so blatant, we feel confident using the word FAKE here.

The aggregate data suggest this problem is completely out of control

Now it’s time to share some deeply troubling numbers.  Looking at the 5,787,573 reviews we’ve collected in the first three months of 2019, we see that:

  • 42% are from Verified Purchasers (2.61m)
  • 58% are from Unverified Purchasers (3.55m)

That’s a lot of Unverified Purchase reviews.  Is this normal? The answer is NO. Compared to the first three months of 2018, we see:

  • 91% are from Verified Purchasers (7.3m)
  • 9% are from Unverified Purchasers (734k)

Here’s a graph to put things in perspective:

Screen Shot 2019-03-27 at 11.21.35 PM

So we’re seeing WAY more Unverified Purchase reviews in 2019.  But wait, there’s more…

This flood of Unverified Purchase Reviews is almost exclusively 5-stars

Of the 3,489,044 Unverified Purchase reviews we’ve collected in the first 3 months of 2019, we see that:

  • 98.2% are 5-star
  • 0.6% are 4-star
  • 0.3% are 3-star
  • 0.3% are 2-star
  • 0.7% are 1-star

For comparison, here’s the distribution of star ratings for the 2,606,178 Verified Purchase reviews from the same time period:

  • 72.6% are 5-star
  • 10.4% are 4-star
  • 4.8% are 3-star
  • 3.7% are 2-star
  • 8.5% are 1-star

Another graph to put things in perspective:

Screen Shot 2019-03-27 at 11.27.12 PM

So it looks like the Unverified Purchase reviews are almost exclusively 5-star.  Has this always been the case? (Spoiler Alert: No) Let’s again compare with the same time period last year.  Here’s the distribution of star ratings for the Unverified Purchases in Q1 of 2018:

  • 72.5% are 5-star
  • 9.2% are 4-star
  • 4.7% are 3-star
  • 4.1% are 2-star
  • 9.5% are 1-star

Looks like the Unverified Purchase reviews weren’t always this insane.  So where did it all go wrong?

The trend shows that the problem is getting much worse

Looking at the monthly trend, we can see that something happened in December, 2018.  Before then, Unverified reviews accounted for less than 20% of all monthly reviews on Amazon.  However we see things shift in December:

Screen Shot 2019-03-28 at 9.10.10 AM

As you can see, the Unverified review problem has been spiraling out of control the last few months.  Not only has the percentage of Unverified reviews skyrocketed, the average rating of those reviews is almost maxed out.  Our data shows that in March, the average star-rating of Unverified reviews is 4.99. (Yes, that means that 99.6% of these reviews are 5-stars).

Screen Shot 2019-03-28 at 9.13.26 AM

But wait!  Could it just be that ReviewMeta doesn’t have the full set of data?

The short answer is no, but it’s worth pointing out some of the shortcomings in our data.  First off, ReviewMeta only collects a sample of Amazon’s data, and that sample isn’t exactly random.  We collect data only on products that visitors run through our system, and it could be argued that our users tend to check products that they already find suspicious in the first place.

That said, our data collection methods have remained the same over the last few years, so comparing the same data over time would nullify any possible skewing.  Also, there’s over 70 million reviews in our 2017-2019 Amazon.com dataset, so that’s A LOT of data.

Ultimately, the shear volume of 5-star, Unverified reviews that we’re seeing recently is completely unprecedented.  Throughout 2017 and 2018, we averaged about 250k 5-star, Unverified reviews per month. However, so far in March 2019, we’ve seen 1.8 MILLION 5-star, Unverified reviews.

Screen Shot 2019-03-28 at 9.14.45 AM

So does this mean I can’t trust any reviews on Amazon?

No, no no – don’t let the big numbers scare you.  While the data paints a troubling picture, the pattern of these fake reviews is very isolated to specific products in specific categories – and very easy to detect, even without using ReviewMeta.

The pattern is usually around 1,000 to 2,000 fake reviews per product.  If there were about 4 million fake reviews created in the last few months, that means there’s roughly 3,000 products that have this pattern.  Considering that Amazon sells more than 600 million products, my guess is that this fake review scheme is only affecting 0.0005% of products on Amazon.

That said, it’s important to keep in mind that these fake reviews are being created specifically to push products up the results for popular searches on Amazon, so the average shopper is much more likely to stumble across one than the 0.0005% figure I just gave you.

How to identify and avoid products that have these fake reviews

On the positive side, these fake reviews are so blatant that it’s very easy to identify them.

My first suggestion, of course, is to run the product through ReviewMeta.com.  Simply copy the product URL and paste it into the search bar at ReviewMeta.com to get a complete analysis on the reviews.  

Screen Shot 2019-03-28 at 10.42.23 AM

If you don’t want to cut and paste/leave Amazon, you can use our Free Browser Extension to see the results of our analysis directly in your browser while shopping.

D2XyMzpUkAARLpp

Even without ReviewMeta, it’s easy to find out if the reviews are created in this pattern.  Simply scroll to the bottom of the page, click the “See all x,xxx reviews” link, then click “Filter By” and select “Verified purchase only”.  You’ll be able to see just how many of those 1,000+ reviews are actual verified purchases.

Screen Shot 2019-03-28 at 10.44.02 AM

 

Screen Shot 2019-03-28 at 10.45.25 AM

When/How is Amazon going to solve the problem??

I reached out to my PR contact at Amazon yesterday afternoon, and at the time of publishing (12:30pm PDT), they have not yet responded.  I’ll post an update if/when they respond.

My experience and the data both show that Amazon is not sitting around twiddling their thumbs.  Our deleted review data shows that Amazon has removed at least 10% of the Unverified, 5-star reviews that were posted in 2019 (though the actual number of deleted reviews is likely higher).

Furthermore, I’ve noticed many examples of these products completely disappearing from Amazon entirely or getting all of their reviews wiped out after a few days, weeks or months.

…Yes, sometimes these products are being sold on Amazon with their thousands of blatantly fake reviews for months before being cleaned up by Amazon.

While Amazon seems to be reacting to the problem, I question what they are going to do moving forward.  The data shows that this problem is getting worse, so (as an outsider using sheer speculation), it doesn’t seem like they’re creating any additional filters to help prevent the problem in the first place.

The fact is that we’ve seen millions of blatantly fake reviews making it through the cracks over the last few months.  Everyone is wondering why it’s so easy for ReviewMeta to detect them in minutes but sometimes taking Amazon months to clean them up.

It seems like now would be a great time for Amazon to completely ban Unverified reviews, which also happens to be the #1 suggestion we had in our article: What Amazon Reviews Should Look Like.