ReviewMeta Analysis Test: Deleted Reviews
At ReviewMeta we have the ability to see how product reviews change over time. One important test we can do with this data is finding deleted reviews. We can detect deleted reviews by checking all the reviews for a product one day, then checking them again at a later date and seeing which ones aren’t there anymore. Although this can seem benign, a large number of deleted reviews can actually indicate that there have been and – possibly still are – unnatural reviews.
There are a few main drivers that lead to reviews being removed:
- The user can remove their review out of their own free will.
- The brand can lobby (either to the reviewer or the platform) to get the review removed – typically negative reviews.
- The platform (i.e. Amazon) may remove a review if they decide that it violates their rules – usually positive reviews.
- The platform can change the rules after the review is posted. For example, many existing Incentivized Reviews were deleted after Amazon updated their policy to prohibit them. Another example is that Amazon lowered the cap on Vine reviews per product and thus deleted many Vine reviews that exceeded that limit.
Reviewers deleting their own reviews on their own accord isn’t necessarily a red flag. A reviewer may feel guilty for writing an overly critical review, or they may not want their username associated with a particular review. Perhaps the product reviewed was something embarrassing. Perhaps they wrote a long ranty review after a few too many glasses of wine but then had regrets the next morning.
Where it gets more dicey is when outside forces coerce a reviewer into removing their own review. Some brands may contact reviewers who leave negative feedback and offer a refund, free product, coupon code, etc. in exchange for the user removing their negative review. If bribing the reviewers isn’t an option, some brands may even threaten negative reviewers with legal action. Even if the legal grounds are completely baseless, most reviewers are normal people who don’t want to risk legal action over an online review and chose instead to bend to the threats and delete their reviews. Even having a brand contact a reviewer might be enough to push a reviewer to delete their review due to privacy concerns. Lastly, brands will sometimes go directly to the reviewing platform and ask for reviews to be removed. They may try to strong-arm the platform into deleting the reviews by leveraging their business relationship, or may even make legal threats on grounds of defamation.
The final, and seemingly most common cause for deletion is that the review platform discovered reviews which violated their terms. Each review platform has their own set of rules designed to keep their reviewers honest and prevent brands from stuffing the ballot. In their effort to create a reliable and accurate resource for their customers they are constantly on the lookout for dishonest reviews. When they discover reviews that break their rules, they will delete them. However the battle between nefarious brands and the reviewing platforms they seek to exploit is a constant cat and mouse game and a platform may not be able to detect all of the sham reviews of a product.
Even though the deleted reviews don’t affect the overall product rating, a high number of deleted reviews can be an indication that the product may have had problematic reviews in the past. These reviews are like mold, just because they have been removed once doesn’t mean more won’t be back the next day. But like all of our tests, the Deleted Reviews test should be considered alongside all of the other tests we conduct on a product’s reviews and reviewers. Failing a the Deleted Reviews test alone isn’t enough to throw out the existing reviews of a product. But if their report card is riddled with failed tests, the Deleted Reviews test can provide evidence that the product’s reviews have been rotten for a long time.
PS – You may be wondering why we don’t show the text, username or title of the deleted reviews, but instead show “REDACTED”. Well, this is for a few reasons. First, we need to respect the platforms for making their decision to remove the review, but also to respect the privacy of the user who might have initiated the deletion.