Browser Extensions 


Shortcomings of

August 15th, 2018

One of the biggest challenges that I’ve faced since launching in 2016 is making it clear what ReviewMeta is and what ReviewMeta is not.

I’ve received an overwhelming amount of thanks and email high-fives for building the tool.  It always brings a smile to my face when someone writes in with kind words. I’m happy to be solving problems for people.

However, I want to be as clear as I possibly can about exactly what problems ReviewMeta is solving, and what it isn’t solving.

ReviewMeta is solving the problem of not having a sufficient amount of organized and compiled information.  I can’t say enough times that ReviewMeta is not able to determine which reviews are “fake” and which are not with 100% accuracy.  We’re simply looking at different trends and making a “best guess” estimate about the reviews.

Here’s the thing: pointing at a review and stating as a matter of fact whether it’s fake or not is impossible.  Amazon removes reviews that they believe are violating their terms.  Any other online tool, informed consumer or magic genie is merely making an educated guess – just like ReviewMeta.

Furthermore, even if you had a magic wand that you could use to point at a review and it would tell you “fake” or “not fake”, you’d only be getting half the picture.

What about seller who screen reviewers before instructing them to leave a review, resulting in a skewed reviewer pool?  There might not be a single “fake” review among them, but intentionally excluding the negative reviews would be misleading, no?

What about reviewers that received a product for free or at a discount for a review?  Is that technically “fake”? Some people say no, others say yes. Even Amazon recently changed their answer from no to yes.

The problem with these review systems is not black and white.  It’s an extremely complex and nuanced issue. Trying to jump to conclusions without looking through the data isn’t a good way to navigate the world of reviews.

The bottom line is this: ReviewMeta is not an infallible magic genie that somehow has all the answers.

What ReviewMeta Is

ReviewMeta is a tool that gives you an estimate, my thought process, and more information about the reviews that are publicly available.    

When I had the idea to create ReviewMeta, I knew that I could only build it under the condition that I would show all of my work in detail on every report.  I did not want to mislead people into thinking that I held that magic wand, and everything I said was a matter of fact.

The goal is to be as transparent as possible and help inform and educate consumers about the data and my methodology.

I’ve intentionally created features to help empower my viewers – such as the ability to change the weights of the tests and recalculate adjusted rating based on your own opinions:

Screen Shot 2018-08-15 at 2.25.18 PM

My mantra is to always provide as much information as possible.  That way, the viewers are able to see the entire story and aren’t told what to think.

A feature that is currently in the works is the Brand Response Dashboard – this will enable brands and sellers to post an official response on their reports – to help spark discussion and debate about the in’s and out’s of the algorithm.  Read more about the Response Dashboard here.

With billions of data points, ReviewMeta cannot possibly get it right every single time.

I sometimes receive emails from reviewers who are offended that they are not at 100% trust, as if I somehow know them on a very personal level and have determined they are not a trustworthy human.  It’s not personal – but I understand that it can be unpleasant to feel falsely labeled as something you are not.

However try thinking of it this way – if you write an article on your blog about cats, and then you google the word “cats” and your own blog post doesn’t come up on the first page, are you offended that Google doesn’t think your content is good?

Of course you aren’t.  Everyone knows that the Google ranking algorithm is a robotic set of rules that tries its best to determine the most relevant content for billions of different possible search terms.  And it’s clear that not every one of their results is perfect.

It would be crazy to think “well, since my blog post about cats is clearly the best information about cats on the internet, and it doesn’t rank #1 for the term ‘cats’ on Google, all of Google must be wrong”.

The same goes for ReviewMeta.  Just because you disagree with one of a billion data points doesn’t really justify throwing the whole thing out.  I recently compared my algorithm against the reviews that ended up removed from Amazon, and found that ReviewMeta is actually a decent predictor of which reviews will eventually be removed from Amazon:

It’s important to always keep in mind what ReviewMeta is and is not while using the tool.  Remember to look through the reports, read the actual reviews and rely on other sources outside of ReviewMeta to ultimately make your purchase decisions.

One response to “Shortcomings of”

  1. Jan Anthonisz says:

    I will in good time give my opinions and comments of Ebay’s illegal and misleading practices

    The matter is now in the hands of the ACCC (Australian Crime and Corrupt Commission) to investigating and possibly closing down Ebay here in Australia for misleading customers with contradictions, hidden fees, false and misleading advertising and much more. [email protected]

Leave a Reply