The Fundamental Problem with Review Clubs
Update 10/3/2016: Amazon BANS Incentivized reviews!
This is Part II of an in-depth series that examines incentivized reviews (reviews with a disclosure similar to: “I received this product for free or at a discount in exchange for my unbiased review”). In Part I, we analyze 7 million reviews and show that incentivized reviews are less critical than non-incentivized reviews.
Let’s put aside the psychology behind reviewing something that you didn’t pay for, and take a closer look at these review clubs.
A common belief is that it’s impossible to review something from a neutral standpoint when you didn’t actually pay for the product, but we’re not here to debate psychology. These review clubs offer the sellers a level of control that wasn’t previously available to them, and we can gain a lot of insight by taking a closer look at how they work.
What is a “review club”
A review club is a 3rd party service that is not affiliated with Amazon, any seller or reviewer, and facilitates the exchange of free or discounted products for reviews. Sellers will typically pay the review club to advertise their products to the reviewers, which are offered for free or at a discount in exchange for a review. There are dozens of review clubs, with these seeming to be the most popular:
Every review club works differently, but all have the same fundamental problems.
Traditionally, reviews were written by customers who discovered the product while shopping for something they wanted, decided to pay retail price, then chose to share their opinion based on the merits or shortcomings of the product, all on their own accord.
Unfortunately, review clubs have completely changed this dynamic. Reviewers are now selected by the sellers to receive products based on their previous review history, are motivated to write reviews in a positive light to maintain a healthy stream of free products, and sometimes get special instructions before writing a negative review (such as contacting the seller first).
Choosing which reviewers get your product is the definition of “selection bias”.
Like we said, every review club operates differently. Some allow sellers to view the actual reviewer profiles before sending the products out, and others just show key stats. Either way, it’s a form of selection bias.
Compare this to sellers who are hoping to generate reviews organically. They don’t have the luxury of picking and choosing which reviewers are worthy of evaluating their products. Anyone can buy and review their product; they don’t get to choose who they get their reviews from.
While many review clubs and sellers claim that they are selecting reviewers who are the most honest, it goes without saying that sellers have a massive financial incentive to get positive reviews instead of negative reviews. It’s hard to trust that they aren’t picking reviewers who tend to give everything 5-stars, especially when the data shows otherwise. Furthermore, since the review clubs are not affiliated with Amazon, and typically generate 100% of their revenue from sellers (it’s usually free for reviewers), their financial incentives are aligned with those of the sellers.
Since there are dozens of review clubs, sellers will ultimately choose to spend their dollars on the ones that can get them the best reviews, creating competition between the review clubs and further motivating them to please the sellers.
Selection bias motivates reviewers to write positive reviews, leading to the rise of mega-reviewers that almost exclusively give positive reviews.
Since sellers are more likely to select reviewers who are less critical, reviewers have an incentive to write positive reviews if they want to continue being selected to receive free or discounted products. We’re finding a lot of power reviewers who have hundreds of reviews and rarely give out critical reviews. Here are some shining examples:
- Alan Gardner
- 5-star: 1,244
- 4-star: 1
- Joshua W
- 5-star: 1,130
- 1-star: 1 (happens to NOT have the incentivized disclaimer)
- Nick
- 5-star: 1,066
- 4-star: 5
- Rachel martin
- 5-star: 1,057
- 4-star: 5 (2 of which happen NOT to have incentivized disclaimer)
- Rachel St. Hilaire
- 5-star: 1,052
- 4-star: 2 (1 of which happens NOT to have incentivized disclaimer)
- 3-star: 1 (happens to NOT have the incentivized disclaimer)
While we aren’t claiming that every single reviewer who has incentivized reviews will follow suit, it does seem to be the “path of least resistance” to keep the stream of free or discounted products flowing smoothly without a fuss from the sellers.
Sellers may give extra instructions such as posting only positive reviews or contacting them before posting a critical review.
Again, this varies with each review club, but the fundamental quid-pro-quo relationship remains the same. Here’s an example from PRN aka Product Review Network that was leaked on Bane-Tech (click here to see the original article)
The email is explicitly instructing reviewers to refrain from posting critical reviews, and to go back and reconsider any critical reviews they have written in the past. It doesn’t get any more blatant than that.
Here’s another example on ReviewKick that was shared via reddit.
Obviously not all sellers are this blatant, but these examples help shed light into what they’re really thinking.
Sellers often harass reviewers for leaving a negative review.
Due to the nature of the exchange, some sellers often believe that they deserve a higher number of stars since they gave the product away for free or at a discount. We’ve seen countless examples of folks complaining on reddit about sellers who harass them for leaving negative feedback. Here are a few examples from the last month:
- Sellers telling you not to leave a neg review
- How would you handle this? (seller is messing with me)
- Seller has been calling me. Now seller has people e-mailing me..
Ultimately, sellers are treating the Amazon review platform as a promotional tool.
No longer is the customer review section for honest customers who want to leave feedback on products they picked; it’s for brands to show off the number of 5-star reviews they acquired from reviewers they picked. Review clubs are the best way to generate reviews that technically aren’t “fake”.
Because some sellers are abusing the system, others are forced to join them to stay competitive.
As we said in Part I of this article series, our data shows that incentivized reviews currently account for the majority of all new reviews on Amazon. Look at the reviews for any top product on Amazon and you’re bound to find plenty of incentivized reviews. More and more sellers are being forced to hop on the bandwagon because of the trend.
So what can we do about the problem of incentivized reviews?
If this trend continues, it will be increasingly difficult to find an honest or a critical review on Amazon. With increasing consumer skepticism, it seems the tides are due for a change.
We look at possible solutions in Part III: What can we do about the problem of incentivized reviews?
Thank you so much for bringing this issue to light. I’ve gotten to the point where I’m not going to consider purchasing any item on Amazon that utilizes this type of marketing. I simply can’t trust the reviews. It’s getting difficult to find items within certain categories that don’t use these review clubs to drum up (positive) reviews. It seems Amazon is complicit in this, since it has been going on for a few years. When I’m searching for something without putting a specific brand name in the search, without fail, the number one seller and the highest rate item within that search is from some no-name company that has 5,000+ reviews, and almost all of them are incentivized reviews. I looked at a container of jojoba oil last night, and it has 1,518 reviews, of which 5 were not incentivized reviews. There were no 1-star reviews, three 2-star reviews, about 20 three-star reviews, and the rest were all 4- and 5-star reviews. It’s getting out of hand.