Browser Extensions 




Unverified Review Problem

Amazon Flooded with Millions of Fake Reviews in 2019

March 28th, 2019

Key findings:

  • Amazon has seen a massive spike in Unverified reviews (reviews without the “Verified Purchaser” badge) in the last few months.  
  • Of the 1.8 million Unverified reviews posted in March 2019, 99.6% are 5-star.  In comparison, during 2017-2018, we saw an average of ~300k unverified reviews per month, only 75% of which were 5-star.
  • These reviews are mostly posted for cheap, off-brand electronic products, and seem to be pushing them onto the first page for popular searches like “iPhone charger”, “bluetooth headphones”, etc.

I’ve honestly never seen it this bad.  It all started when I noticed an increase in complaints on Twitter about cheap electronic products on Amazon that were flooded with blatantly-fake 5-star reviews.  They all seem to follow the same pattern:

  1. Off-brand, cheap electronic item (headphones, chargers, cables, etc)
  2. Literally 1,000’s of reviews
  3. Nearly 100% unverified purchases
  4. Nearly 100% 5-star reviews
  5. All reviews created over just a few days
  6. Reviews for dozens of different random “variations”, such as “Color: MC0326-C2”, “Color: Blue-42”, etc.

Screen Shot 2019-03-28 at 8.56.36 AM

Screen Shot 2019-03-28 at 8.58.22 AM

It’s not like these products were hidden deep in the depths of Amazon’s catalogue – this fake review scheme seems to be pushing these products to the first page of search results. We recently posted an article showing that 10 of 22 results for the search term “iPhone Charger” were products that had this same pattern of fake reviews.

At the time of writing, this off-brand MacBook Pro Charger has the “Amazon’s Choice” badge, even though the product has thousands of blatantly fake reviews.  

Screen Shot 2019-03-27 at 5.44.28 PM

Our analysis of this product throws out every single one of the 2,855 reviews.

Screen Shot 2019-03-27 at 5.46.26 PM

If you’ve spent any time on ReviewMeta, you’ll know that we are very reluctant to use the word “fake”.  However these reviews are so blatant, we feel confident using the word FAKE here.

The aggregate data suggest this problem is completely out of control

Now it’s time to share some deeply troubling numbers.  Looking at the 5,787,573 reviews we’ve collected in the first three months of 2019, we see that:

  • 42% are from Verified Purchasers (2.61m)
  • 58% are from Unverified Purchasers (3.55m)

That’s a lot of Unverified Purchase reviews.  Is this normal? The answer is NO. Compared to the first three months of 2018, we see:

  • 91% are from Verified Purchasers (7.3m)
  • 9% are from Unverified Purchasers (734k)

Here’s a graph to put things in perspective:

Screen Shot 2019-03-27 at 11.21.35 PM

So we’re seeing WAY more Unverified Purchase reviews in 2019.  But wait, there’s more…

This flood of Unverified Purchase Reviews is almost exclusively 5-stars

Of the 3,489,044 Unverified Purchase reviews we’ve collected in the first 3 months of 2019, we see that:

  • 98.2% are 5-star
  • 0.6% are 4-star
  • 0.3% are 3-star
  • 0.3% are 2-star
  • 0.7% are 1-star

For comparison, here’s the distribution of star ratings for the 2,606,178 Verified Purchase reviews from the same time period:

  • 72.6% are 5-star
  • 10.4% are 4-star
  • 4.8% are 3-star
  • 3.7% are 2-star
  • 8.5% are 1-star

Another graph to put things in perspective:

Screen Shot 2019-03-27 at 11.27.12 PM

So it looks like the Unverified Purchase reviews are almost exclusively 5-star.  Has this always been the case? (Spoiler Alert: No) Let’s again compare with the same time period last year.  Here’s the distribution of star ratings for the Unverified Purchases in Q1 of 2018:

  • 72.5% are 5-star
  • 9.2% are 4-star
  • 4.7% are 3-star
  • 4.1% are 2-star
  • 9.5% are 1-star

Looks like the Unverified Purchase reviews weren’t always this insane.  So where did it all go wrong?

The trend shows that the problem is getting much worse

Looking at the monthly trend, we can see that something happened in December, 2018.  Before then, Unverified reviews accounted for less than 20% of all monthly reviews on Amazon.  However we see things shift in December:

Screen Shot 2019-03-28 at 9.10.10 AM

As you can see, the Unverified review problem has been spiraling out of control the last few months.  Not only has the percentage of Unverified reviews skyrocketed, the average rating of those reviews is almost maxed out.  Our data shows that in March, the average star-rating of Unverified reviews is 4.99. (Yes, that means that 99.6% of these reviews are 5-stars).

Screen Shot 2019-03-28 at 9.13.26 AM

But wait!  Could it just be that ReviewMeta doesn’t have the full set of data?

The short answer is no, but it’s worth pointing out some of the shortcomings in our data.  First off, ReviewMeta only collects a sample of Amazon’s data, and that sample isn’t exactly random.  We collect data only on products that visitors run through our system, and it could be argued that our users tend to check products that they already find suspicious in the first place.

That said, our data collection methods have remained the same over the last few years, so comparing the same data over time would nullify any possible skewing.  Also, there’s over 70 million reviews in our 2017-2019 Amazon.com dataset, so that’s A LOT of data.

Ultimately, the shear volume of 5-star, Unverified reviews that we’re seeing recently is completely unprecedented.  Throughout 2017 and 2018, we averaged about 250k 5-star, Unverified reviews per month. However, so far in March 2019, we’ve seen 1.8 MILLION 5-star, Unverified reviews.

Screen Shot 2019-03-28 at 9.14.45 AM

So does this mean I can’t trust any reviews on Amazon?

No, no no – don’t let the big numbers scare you.  While the data paints a troubling picture, the pattern of these fake reviews is very isolated to specific products in specific categories – and very easy to detect, even without using ReviewMeta.

The pattern is usually around 1,000 to 2,000 fake reviews per product.  If there were about 4 million fake reviews created in the last few months, that means there’s roughly 3,000 products that have this pattern.  Considering that Amazon sells more than 600 million products, my guess is that this fake review scheme is only affecting 0.0005% of products on Amazon.

That said, it’s important to keep in mind that these fake reviews are being created specifically to push products up the results for popular searches on Amazon, so the average shopper is much more likely to stumble across one than the 0.0005% figure I just gave you.

How to identify and avoid products that have these fake reviews

On the positive side, these fake reviews are so blatant that it’s very easy to identify them.

My first suggestion, of course, is to run the product through ReviewMeta.com.  Simply copy the product URL and paste it into the search bar at ReviewMeta.com to get a complete analysis on the reviews.  

Screen Shot 2019-03-28 at 10.42.23 AM

If you don’t want to cut and paste/leave Amazon, you can use our Free Browser Extension to see the results of our analysis directly in your browser while shopping.

D2XyMzpUkAARLpp

Even without ReviewMeta, it’s easy to find out if the reviews are created in this pattern.  Simply scroll to the bottom of the page, click the “See all x,xxx reviews” link, then click “Filter By” and select “Verified purchase only”.  You’ll be able to see just how many of those 1,000+ reviews are actual verified purchases.

Screen Shot 2019-03-28 at 10.44.02 AM

 

Screen Shot 2019-03-28 at 10.45.25 AM

When/How is Amazon going to solve the problem??

I reached out to my PR contact at Amazon yesterday afternoon, and at the time of publishing (12:30pm PDT), they have not yet responded.  I’ll post an update if/when they respond.

My experience and the data both show that Amazon is not sitting around twiddling their thumbs.  Our deleted review data shows that Amazon has removed at least 10% of the Unverified, 5-star reviews that were posted in 2019 (though the actual number of deleted reviews is likely higher).

Furthermore, I’ve noticed many examples of these products completely disappearing from Amazon entirely or getting all of their reviews wiped out after a few days, weeks or months.

…Yes, sometimes these products are being sold on Amazon with their thousands of blatantly fake reviews for months before being cleaned up by Amazon.

While Amazon seems to be reacting to the problem, I question what they are going to do moving forward.  The data shows that this problem is getting worse, so (as an outsider using sheer speculation), it doesn’t seem like they’re creating any additional filters to help prevent the problem in the first place.

The fact is that we’ve seen millions of blatantly fake reviews making it through the cracks over the last few months.  Everyone is wondering why it’s so easy for ReviewMeta to detect them in minutes but sometimes taking Amazon months to clean them up.

It seems like now would be a great time for Amazon to completely ban Unverified reviews, which also happens to be the #1 suggestion we had in our article: What Amazon Reviews Should Look Like.


  • Quagmire

    Funny how some reviews are allowed to stay and others are not. I had a few reviews deleted because I called out the sellers for having all 5 star reviews on the same date. I assume they complained about it and had my reviews deleted. Now on this site I see my name listed as having previous reviews deleted! lol

  • Barry Melius

    Amazon is always tweaking their algorithms as part of what I assume is ongoing A/B testing. When you click See all reviews at the bottom of the reviews on the main product page you are switched to another page. The default setting for this page normally is All reviewers but for a couple of weeks a while ago they had the default set at Verified purchase only. I personally preferred this, but it also eliminated all the reviewers that Amazon furnishes free copies (this can be a significant number) and they put a lot of work in for free and could not have been happy about this decision. There are also a lot of independent reviewers that receive promo copies from the publishers or their proxies but none the less attempt to write honest reviews. Having reviews posted early must have a positive impact on sales and Amazon is in the business of making money. So to eliminate all unverified purchase reviews would impact Amazon’s bottom line. I can’t imagine Amazon doesn’t care about review credibility, but its a hard choice for them to make.

  • Mike

    Is there any chance you might be able to modify the browser extensions to insert data into the Amazon product page itself? I like the convenience of the ReviewMeta rating being easily visible on the menu bar icon, but I honestly forget to look up there at times…

    I’m thinking something along the lines of how the extension for Keepa works. They insert a chart of the price history right into the product page. Ever consider this?

    • This is in the works right now. The challenge is that we want the extension to be minimally intrusive as to not annoy people, but also as helpful as possible. Keeping these two goals in mind is bogging down the design process at the moment.

  • Xagec Xagecoff

    What’s the problem them ratings being based entirely/predominantly on verified purchases?

    • We’re asking the same question. Seems like Amazon allowing unverified reviews is causing more harm than good at this point.

  • Mike in Oceanside California

    Well I see HOW they do it since below is just one of over 10-20 emails I get EVERY single NIGHT from Chinese SCAMMERS begging me to write reviews. MOST offer me large KICKBACK for the trouble as well. Report them to Amazon but they could care less. They even BANNED me from any reviews due to me constantly telling people what was going on in MY reviews…Then they literally removed 15 years of ALL my reviews to boot so I am done with THEM as well. They don’t care at all it seems but man do they ever want those sales revenues no matter HOW they get them. It sickens me but at least I have to my amazement EASILY transitioned to buying everything from Target, Walmart & Best Buy etc and found it often cheaper anyway and I get reviews I can trust! Man, you want a shock? Check reviews for items at Best Buy VS Amazon on TV sets etc and see the difference! A real eye opener. Amazon has simply proven itself 100% irrelevant to me.

    free product for testing
    Trash
    x
    [email protected]
    x
    子非鱼
    Wed, Mar 20, 12:48 AM (1 day ago)
    to jxxxxxx

    You have blocked [email protected] New messages from this sender will be sent to Spam.
    Unblock senderMove to spam
    Dear USMC Combat Vet :
    My name is Sally .
    We are a sanitaryware manufacture as well as the Amazon seller.
    In here, I am have some good news for you .
    If you are Amazon member and meet the below requirement,you can get this free new handheld bidet sprayer for testing in the Amazon website .(see the below photo)
    The buyer can not leave more than 2 reviews for 7 consecutive days in their profile and no more than 8 reviews within 30 consecutive days.Meanwhile the review rate can not be too high whih controls below 15-25% is the best.
    The buyer searchs the product by keyword bidet sprayer and the product is at the first page.Shop around and browse each product for at least 30 seconds.Add the product into a wish shopping cart .Placing an purchase order in the next day ,buy our product only.
    After giving the product review ,send the proof to the seller and the full refund will be processed through PayPal (the exchange rate of the day and includes PayPal 4% handling fee).
    Limited quotas, first come first served.Let us know what you thing ,so that we can move on.
    Looking forward to your early reply.
    Best regard,
    Sally

  • OnlineBiz22

    This certainly highlights bad actors among the Amazon marketplace.

    One thing I’m wondering about is the assigning of “Verified Purchase” tags. Amazon doesn’t always give that to every purchase, even when it’s for an item paid for on Amazon. For example, if I buy something with a discount code, Amazon may remove the Verified Purchase tag for the review I write of that item. How do you think this plays into your observations. We don’t know the percentage affected, but some of these non-verified reviews certainly are a result of this situation.

    Also, where/how do you get access to all the Amazon review data that you reference in this and other posts?

    • This dataset is what we’ve acquired after scanning hundreds of millions of Amazon reviews.

  • andrea

    Well, what can I say… Is the Pope Catholic?