Browser Extensions 


ReviewMeta Analysis Test: Unverified Purchases

April 27th, 2016

The verified purchaser system is simple. If you purchase a product from an online vendor and then go on to review it, the vendor will add a flair to your review saying “verified purchaser”. Since most sites will allow users to post reviews whether or not they have purchased the product from the site, the verified purchaser flair adds another level of authenticity and legitimacy to those reviews. When you read a review with a verified purchaser flair you know that the reviewer received the product from the vendor.  This is one of many safeguards vendors employ in order to protect their customers by stopping some brands from artificially inflating their product’s rating and reputation.  

Regardless, you’ll still see some brands stuffing a disproportional amount of unverified reviews in order to boost their rating.  While some amount of unverified reviews are to be expected for any product, it can be a red flag if their percentage is too high, or if the unverified reviews are telling a story that is vastly different from the verified reviews.

Although we know that verified reviewers most likely received the product, the flair alone is not an indication of a trustworthy review. Some brands will select reviewers and give them a discounted or free product in exchange for an “honest review”. Despite being free or heavily discounted, some sites may still label these reviews as verified reviews.  Furthermore, some brands will offer a “trial size” or generate coupons or gift codes for 100% off, allowing some to achieve the “verified purchaser” flair on fraudulent reviews. We have even heard of brands going so far as to create fake distribution centers where they route all fake orders that never get shipped out.

At ReviewMeta we are able to learn more about a product’s reviews by examining the data that the verified purchaser system creates. While it’s perfectly normal to see some unverified reviews, an excessive percentage can trigger a warning or failure for this test . Furthermore, if the unverified reviews have a higher average rating than verified reviews, we’ll check to see if this discrepancy is statistically significant.  This means that we run the data through an equation that takes into account the total number of reviews along with the variance of the individual ratings and tells us if the discrepancy is more than just the result of random chance.  (You can read more about our statistical significance tests here).  If the unverified reviews have a significantly higher average rating than verified reviews, it’s a strong indicator that the unverified reviews are influencing the overall rating. This would be a strong indicator that these reviewers are not benign, but in fact, abusing the system.

  • K.G.

    Dear ReviewMeta team,

    Can you please advise whether the Amazon Vine review process is taken into account when classifying reviews as unverified?

    If Vine reviews form a subset of the unverified reviews, is it possible to identify what percentage of the unverified reviews stem from the Vine review process?

    Thank you


    • We actually consider Vine reviews to be verified even though Amazon does not. The reason is that we know they received the product, and the unverified test is really looking at reviews that cannot be verified in any way. The report does not show the percentage of vine reviews, but that information can be available upon request.

      • K.G.

        Thank you for your response. Please provide me an email address to which I can send specific feedback. I have screen shots from reviewmeta and from which I believe you will be interested in. It may not be prudent to put this level of information up through this blog. I am happy to share the information with you offline. Thank you, regards, K.G.